Web Development with Python & Django

Announcements

  • Assignment 5 due April 13th
  • I will get your assignment 4 grades back this week.

We are going to learn how to build web applications with Python

  • Applications run on a web server
  • The webserver uses a protocol—hyper text transfer protocol (HTTP)—to communicate with clients
  • It listens for HTTP requests and return responses
  • The responses are often dynamically generated in response to the request

The internet is layers on top of layers

Anatomy of the HTTP request/response cycle

  1. Web browser makes a request to a server for a URL
  2. Server parses request, generates a response to the request and returns it to the browser
  3. Browser parses the response, renders the page, and downloads any media files (CSS, JavaScript, or images) referenced in the HTML

An HTTP Request

I visit http://example.com/some-url/,
the request includes:

  • The protocol (http/https)
  • The hostname (example.com)
  • The path (/some-url/)
  • The method (GET/POST/Etc.)
  • Headers
    • User agent
    • Referrer
    • *Username/password
    • *Cookies
  • *Form content

An HTTP Response

The server's response includes:

  • A status code
    • 2xx - request was successful
    • 3xx - redirection: further action needs to be taken
    • 4xx - client error
    • 5xx - server error
  • Headers
    • Content type server is returning (text/html)
    • Size of the response
    • *How long the content can be cached for
    • *Cookies to set
  • The body of the response (i.e. the HTML itself)

A little more on the common status codes

  • 200 - OK
  • 301 - moved permanently
  • 302 - moved temporarily
  • 401 - unauthorized
  • 404 - not found
  • 500 - generic server error

What does a web application do?

In a nutshell: receive HTTP requests and generate HTTP responses

It's generally a long-running python process (or multiple processes) that sits there waiting for HTTP requests so it can return a response.

How do you make a web application?

You could write everything from scratch (not advised)

Or you could use a web application framework

What does a web application framework do?

At minimum:

  • Parse an HTTP request into its constituent pieces
  • Provide a way to map URLs (or URL patterns) to code that can handle them
  • Provide a way to generate HTML (templating)
  • Provide a way to create an HTTP response

What else might a web application framework do?

  • Provide a way to store state (most commonly in a database)
    • Connect to a database (DB)
    • Generate DB queries
    • Translate from DB query to Python objects
    • Translate Python objects to a DB query to store them
  • Provide a way to generate HTML forms/validate input
  • Provide a way to authenticate users
  • Handle caching
  • Ensure security
  • Send emails

Typically these are called full-stack web application frameworks

Django is one such full-stack Python framework

  • A free, open-source full-stack web framework
  • Created in 2003, open-sourced in 2005
  • Latest release is 4.0.3 (March 1, 2022)
  • Most common full-stack framework for Python
  • Includes all the components we talked about (and more)
  • Designed to be flexible: use the pieces you need, ignore the ones you don't.
  • Has many plugins you can install to extend it even further

Key Django Concepts

  • Project: a single web application (often project = website)
  • App: a single unit of functionality (one project can contain multiple apps). Apps should do one thing and do it well.
  • Router: code that maps URLs to views that handle them
  • View: function (or class) that receives an HTTP request as its argument and returns an HTTP response
  • Model: a python class that maps to a database table

Why do we need a database?

In a word: state. Web applications need a place to store things.

What kinds of things?

Websites will usually provide resources

  • An e-commerce site will have a catalog of items, with names, descriptions, prices, dimensions, etc.
  • A news site will have articles, sections, videos, and photo galleries
  • A banking website will have accounts, balances, statements

All this "stuff" has to be stored somewhere. This is called the application's resource state.

The other type of state that is common in web apps is called application state:

  • A user signs into a site and the site remembers they have signed in for subsequent page visits.
  • A user adds and item to their cart in an e-commerce site, then goes to a checkout page; the item is still there in their cart
  • A user comes back to a site after a few days, the site remembers who they are and shows them new things it thinks they might like
  • Application state is how we remember how users have interacted with our app.

An app might have resource state, but not have a concept of users, and thus no application state.

HTTP is a stateless protocol

  • There is no link between two requests being successively carried out on the same connection
  • How does something like you Amazon shopping basked work then?
  • Cookies!

Cookies to the rescue

  • A cookie is data that a server sends to a user's web browser.
  • The browser stores the cookie and sends it back to the same server with later requests.
  • Think of it as a unique name-tag the server pins on your browser
  • Cookies are small. Max size: 4KB
  • This is why application state needs to be stored on the server.

Let's talk about state some more, resource or application state...

Idea 1: Just keep it all in memory in python objects

  • If the server crashes everything is lost.
  • The may be too much resource state to fit in memory.
  • If we load-balance requests among N servers, then we have to keep N copies of everything and find ways to synchronize it.

Idea 2: Just keep it all in a file

  • How do we convert what's in the file to Python objects?
  • How do we find a specific resource in the file fast enough?
  • We probably can't create or update resources without re-writing the whole file.
  • If we load-balance requests among N servers, we have to find ways to synchronize the file.

What we need is:

  • A way to store structured information on disk that can be translated to/from Python in-memory objects easily
  • A way to create, update, or delete individual resources easily and quickly
  • A way to query resources based on their properties
  • A way to store resources separately from a web server so multiple servers can all have access to the same resources
  • This is exactly what databases were designed for!
  • There are different kinds of databases
  • Django is intended to be used with a relational database
  • Relational databases store resources in tables
  • You can think of a database table as being kind of like a Pandas Dataframe
  • Relational databases have their own query language: SQL
    • Django can generate SQL for us
    • And translate table rows to Python objects
    • Or translate Python objects to table rows
  • Databases often run on a separate server of their own

Next up...

Let's create a Django project of our own.