During our third week, we have been finishing up Part 1 of our first project, which has us designing a database with data capture or OLTP (online transaction processing) in mind. This initial design of our database relies primarily on what we have learned last week about normalizing databases to reduce data redundancy. By contrast, this week’s readings have focused on OLAP (online analysis processing), which serves a different purpose than OLTP. My understanding of these differences are outlined in the table below:

OLTP	OLAP
Purpose is operational record keeping	Purpose is analytical decision making / evaluation of performance (answer how/what/why questions)
Optimized to process transactions quickly — usually processes one transaction at a time	Must handle up to hundreds of thousands of transactions at a time
Updated to maintain the current state (no historical records are kept)	Historical records are important to provide context for the evaluation of performance over time

OLTP and OLAP differ fundamentally because the goals of the end user are different. While OLTP typically use relational databases that are normalized (typically in the third normal form) to reduce data redundancy and to be able to process single transactions quickly, OLAP makes use of dimensional modeling to deliver data that is understandable and appealing to the end user.

Kimball and Ross (2013) use a restaurant metaphor to describe one form of OLAP architecture — Kimball’s Data Warehouse/ Business Intelligence (DW/BI) architecture .

In the restaurant metaphor, the kitchen is the staging area of the DW/BI. Just as kitchen staff must ensure that the ingredients they procure are high quality and are prepared in a manner that suits the diner’s palate, we who design the data staging area are concerned with cleaning the data (e.g. by fixing misspellings or formatting inconsistencies) that we extract from multiple sources. This process is called the ETL (Extract, Transformation, and Load) System.

Just as we would not expect diners to wander into the back kitchen, we would not expect users to have access to the staging area / ETL. Diners belong in the dining room, where they are presented with prepared food. A DW/BI user’s domain is in the presentation area, where they should be presented with data in a meaningful and easy-to-understand format.

---

References

Kimball, R. and Ross, M. (2013). The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling (3rd Ed.). Indianapolis, IN: John Wiley and Sons.

Ross, M. (2004). Differences of Opinion. Kimball Group. https://www.kimballgroup.com/2004/03/differences-of-opinion/

This week we learned about grouping results (summaries) and subqueries before moving on to database design.

Summaries

When creating summary queries, you use one or more aggregate functions: AVG, SUM, MMIN, MAX, COUNT (or COUNT (*) ). Any SELECT clause having an aggregate function CANNOT have a non-aggregate column UNLESS it is used to GROUP results in a GROUP BY clause. As an example, you can display a SUM OF INVOICES (aggregate column) GROUPED BY vendor name (non-aggregate column).

Syntax is very important when crafting summary queries. The GROUP BY clause needs to occur before the search condition in the HAVING clause and the order by list in the ORDER BY clause to produce the intended results of a summary query.

Subqueries

Most JOINs can be restated as a subquery and vice versa.

Subqueries can be coded in an outer SELECT statement in a:

• WHERE clause as a search condition
• HAVING clause as a search condition
• FROM clause as a table specification
• SELECT clause as a column specification

Subqueries can return a:

• single value (generally used in WHERE and HAVING)
• list of values, i.e. one column (generally used the IN of a WHERE / HAVING clause or in a SELECT clause
• table of values, i.e. multiple columns (generally used in FROM)

JOINs versus Subqueries

JOINS	SUBQUERIES
SELECT clause of JOIN can include columns from both tables.	Can pass an aggregate value to main query
more intuitive when existing relationships between tables (primary and foreign keys)	more intuitive when forming an ad hoc relationship between tables
	long/complex queries sometimes easier to code/read

• both used for queries that interact with 2+ tables
• usually they translate back and forth

Database Design

An important aspect of database design is normalization–separating data in a data structures to separate and related tables to reduce data redundancy. However, in the real world, databases are not completely normalized. Out of the seven normal forms outlined by the Murach text, only the first, second, and third normal forms are generally used in the real world.

Below is the first draft of my team’s database design for a system that will be able to give users the ability to search for a teacher’s schedule (the classes they teach, where they teach them, and when they teach them).

Today marks the first official day of CST 363, a Databases course taught with MySQL and Python 3. Because I am returning to work next week and because the professor posted our weekly work in advance, I made sure to get a head start to manage my time effectively. Therefore, I have already completed all of Week 1’s readings and the assignments.

I can remember the term CGI as far back as I started using the Internet back in the mid to late 90s on AOL. I recall seeing member pages with a directory named cgi-bin. Now I actually know what CGI (Common Gateway Interface) is and what the cgi-bin directory is for. A Common Gateway Interface allows webservers to execute applications and create webpages dynamically. The cgi-bin directory is the directory where such programs must be stored (on a webserver) in order to be executed correctly by the webserver.

This week, I was able to created a web-based form that simulating either a user attempting to login or to register on a server. First, I ran Python script (provided my the professor) that started an HTTP server on my local machine. An HTTP server allows you to access and HTML file on your browser via a GET request to that server:

127.0.0.1 - - [09/Jan/2019 18:02:09] "GET /login.html HTTP/1.1" 200 -

As previously mentioned, our assignment was to create a web-based form to allow a user to login or register. A user’s login information (userid and password) had to be checked against a MySQL database in order to determine if the credentials were valid for either registration (userid can’t already exist) or logging in (password must match for an existing userid).

The HTML <form> tag requires two attributes: action and method. The action is the name of a file or the name of a program the HTTP server should send form-data to. The method specifies whether the HTTP server should send the form-data with a GET or POST request. In this case, our web form sent a POST request encoding key-value pairs (userid: ‘your_user_id’ and password: ‘your_password’) to a Python 3 program that would use this data to authenticate the credentials (or insert in the case of valid registration) against our MySQL database.

127.0.0.1 - - [09/Jan/2019 18:53:04] "POST /cgi-bin/login.py HTTP/1.1" 200 -