Part 1 - What is a webserver? And what is a web application?

Before we jump into building the necessary components for our little FastAPI clone, let’s spend some time to define the basic entities that are involved in making things work on the web.

Webserver

A webserver is a program that runs on a server, listens for incoming requests from clients and tries to send back useful responses.

graph LR client-->|request|webserver webserver-->|response|client

The “web” part of the server is really just the fact that it speaks one of the usual “languages”/protocols that are used to exchange resources on the internet. In almost all cases this means that it speaks some version of HTTP (we’ll go into detail on HTTP in a later post).

graph LR client---|HTTP|data subgraph webserver http_parser[HTTP parser] data[request/response] data---http_parser end

Good webservers are very efficient at scaling up and down in the face of changing numbers of incoming requests and can run reliably for long times. Usually they also take care of jobs like decrypting/encrypting traffic or decompressing/compressing requests/responses. All in all they are in charge of the raw mechanics of communication, that’s what they are written and optimized for. They don’t care or know about the content of the messages they are processing.

Static websites

In the early days of the internet, webservers mostly just responded with static files (e.g. HTML, CSS, images, …) that were sourced right from the filesystem. So a request might come in for /about.html and the webserver would look in some specific local directory for /about.html and send it right along as the response body. If the requested resource was not available, it would reply with a 404 error.

sequenceDiagram client->>webserver: GET /about.html webserver->>filesystem: read /about.html filesystem->>webserver: ...content... webserver->>client: 200 OK ...content... client->>webserver: GET /cats.html webserver->>filesystem: read /cats.html filesystem-->>webserver: unavailable webserver->>client: 404 NOT FOUND

That is a very static situation. The webserver has no means to dynamically alter the resource it is sending back to you. For many kinds of websites this is totally sufficient (this blog is an example). Other websites might want to create a more dynamic response. This could e.g. be based on whether you are logged in or on some additional data you sent with the request or by injecting data from a database.

The question was: how do you go from a webserver that was initially just designed to reply super efficiently with static resources, to a webserver that can respond in a much more dynamic way?

Web application

It’s important to understand why this isn’t a trivial problem to solve. At that time, many webservers were written in C with few choices for dynamically extending their functionality. You might argue that one could have just written dynamic endpoints in C and then somehow registered them in the webserver. But those webservers weren’t really designed for that and it’s not actually their job, as discussed earlier in the post. Also C is not the friendliest of languages, an interpreted language (without the compilation-cycle) might be preferable.

So there were a couple of desired features: - be able to use existings webservers and put your dynamic web functionality behind them - separation of responsibilities between webserver and web application - webserver: reliably and efficiently handle the request/response mechanics and speak some web protocol (e.g. HTTP) - web application: provide dynamic behavior for certain endpoints - be able to use different programming languages in webserver and web application

In essence you wanted something like the following, where the web application sits behind the webserver and can provide specific dynamic behavior.

graph LR client---|HTTP|webserver subgraph server static[filesystem] webapp[web application] webserver---|static|static webserver---|dynamic|webapp end

The most interesting aspects of the relationship between webserver and web application are usually the way the webserver triggers the web application and the way the communication between them is facilitated.

Web framework

Often you’ll also hear the term “web framework”. This is simply a library that allows you to more quickly develop web applications. It takes care of the nitty-gritty details of providing you with useful request/response objects, providing ways to handle errors, providing ways to register your endpoints and so on. If you look at the amount of work that 3 lines of code in FastAPI can do, you understand that something is doing some heavy lifting in the background. And that something is the web framework.

@app.get("/")
def hello_world():
    return {"hello": "world"}

This application must check incoming requests. If they are GET requests to the root path / it must call the function. It must create a correct response with an HTTP status code and headers set. It must set the correct mimetype and serialize the python dict as JSON into the response body. And so on, and so on.

FastAPI is an example for a web framework, but it itself is based on a lower-level web framework called starlette. The flask library is another example for a web framework.

In short: web frameworks allow you to develop web applications more quickly by providing a high level of abstraction around the request/response cycle, and letting you focus on the actual logic of the dynamic behaviors you want to provide.

Notes

I found this answer on stackoverflow to be a great, intuitive overview of what a webserver and a web application are and why they should be separated. Maybe it can give you an additional perspective on the issue at hand.

In the next post we’ll look at different solutions for letting webservers talk to python web applications that have evolved since the 90s.