Software Design by Example: Serving Web Pages

class: slide-title

<p>
  <a href="https://third-bit.com/sdxpy/">Software Design by Example</a>
</p>
<h1>Serving Web Pages</h1>
<div class="bottom">
  <a href="../">chapter</a>
</div>

---

## The Problem

-   Uploading and downloading files (<a class="x-ref" href="../../ftp/">Chapter 21</a>) is useful,
    but we want to do more

-   Don't want to create a new <a class="gl-ref" href="../../glossary/#gl:protocol" title="A set of rules that something promises to obey, i.e., the contract between that thing and its users." markdown="1">protocol</a> for every interaction

-   Use a standard protocol in a variety of ways

---

## HTTP

-   <a class="gl-ref" href="../../glossary/#gl:http" title="The protocol used to exchange information between browsers and websites, and more generally between other clients and servers. Communication consists of requests and responses." markdown="1">Hypertext Transfer Protocol (HTTP)</a> specifies
    what kinds of messages clients and servers can exchange
    and how those messages are formatted

-   Client sends a <a class="gl-ref" href="../../glossary/#gl:http_request" title="A precisely-formatted block of text sent from a client such as a browser to a server that specifies what resource is being requested, what data formats the client will accept, etc." markdown="1">request</a> as text over a socket connection

-   Server replies with a <a class="gl-ref" href="../../glossary/#gl:http_response" title="A precisely-formatted block of text sent from a server back to a client in reply to a request." markdown="1">response</a> (also text)

-   Requests and responses may carry (non-textual) data with them

-   *Server can respond to requests however it wants*

---

## HTTP Requests

-   A <a class="gl-ref" href="../../glossary/#gl:http_method" title="The verb in an HTTP request that defines what the client wants to do. Common methods are `GET` (to get data) and `POST` (to submit data)." markdown="1">method</a> such as `GET` or `POST`

-   A <a class="gl-ref" href="../../glossary/#gl:url" title="A multi-part identifier that specifies something on a computer network. A URL may contain a protocol (such as `http`), a hostname such as `example.com`, a port (such as 80), a path (such as `/homepage.html`), and various other things." markdown="1">URL</a>

-   A <a class="gl-ref" href="../../glossary/#gl:http_protocol_version" title="Specifies the version of HTTP being used, which in turn defines what headers can appear, how they are to be interpreted, etc." markdown="1">protocol version</a>

```txt
GET /index.html HTTP/1.1
```

---

## Headers

-   Requests may also have <a class="gl-ref" href="../../glossary/#gl:http_header" title="A name-value pair at the start of an HTTP request or response. Headers are used to specify what data formats the sender can handle, the date and time the message was sent, and so on." markdown="1">headers</a>

```txt
GET /index.html HTTP/1.1
Accept: text/html
Accept-Language: en, fr
If-Modified-Since: 16-May-2023
```

-   A key can appear any number of times

---

## HTTP Response

-   Protocol and version

-   A <a class="gl-ref" href="../../glossary/#gl:http_status_code" title="A numerical code that indicates what happened when an HTTP request was processed, such as 200 (OK), 404 (not found), or 500 (internal server error)." markdown="1">status code</a> and phrase

-   Headers, possibly including `Content-Length` (in bytes)

-   Blank line followed by content

```txt
HTTP/1.1 200 OK
Date: Thu, 16 June 2023 12:28:53 GMT
Content-Type: text/html
Content-Length: 53

<html>
<body>
<h1>Hello, World!</h1>
</body>
</html>
```

---

## Requests

```py
import requests

response = requests.get("http://third-bit.com/test.html")
print("status code:", response.status_code)
print("content length:", response.headers["content-length"])
print(response.text)
```

```
status code: 200
content length: 103
<html>
  <head>
    <title>Test Page</title>
  </head>
  <body>
    <p>test page</p>
  </body>
</html>
```

---

## HTTP Lifecycle

---

## Basic HTTP Server

```py
from http.server import BaseHTTPRequestHandler, HTTPServer

PAGE = """<html><body><p>test page</p></body></html>"""

class RequestHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        content = bytes(PAGE, "utf-8")
        self.send_response(200)
        self.send_header(
            "Content-Type", "text/html; charset=utf-8"
        )
        self.send_header("Content-Length", str(len(content)))
        self.end_headers()
        self.wfile.write(content)

if __name__ == "__main__":
    server_address = ("localhost", 8080)
    server = HTTPServer(server_address, RequestHandler)
    server.serve_forever()
```

---

## Running the Server

```sh
python basic_http_server.py
```

-   Displays nothing until we go to `http://localhost:8080` in our browser

-   Browser shows page

-   Shell shows log messages

```
127.0.0.1 - - [16/Sep/2022 06:34:59] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [16/Sep/2022 06:35:00] "GET /favicon.ico HTTP/1.1" 200 -
```

---

## Serving Files

```py
class RequestHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        try:
            url_path = self.path.lstrip("/")
            full_path = Path.cwd().joinpath(url_path)
            if not full_path.exists():
                raise ServerException(f"{self.path} not found")
            elif full_path.is_file():
                self.handle_file(self.path, full_path)
            else:
                raise ServerException(f"{self.path} unknown")
        except Exception as msg:
            self.handle_error(msg)
```

---

## Read and Reply

-   Translate path in URL into path to local file

-   <a class="gl-ref" href="../../glossary/#gl:path_resolution" title="The process of converting the filename portion of a URL into a specific file on disk." markdown="1">Resolve</a> paths relative to server's directory

```py
def handle_file(self, given_path, full_path):
    try:
        with open(full_path, 'rb') as reader:
            content = reader.read()
        self.send_content(content, HTTPStatus.OK)
    except IOError:
        raise ServerException(f"Cannot read {given_path}")
```

---

## Handling Errors

```py
ERROR_PAGE = """\
<html>
  <head><title>Error accessing {path}</title></head>
  <body>
    <h1>Error accessing {path}: {msg}</h1>
  </body>
</html>
"""
```

```py
def handle_error(self, msg):
    content = ERROR_PAGE.format(path=self.path, msg=msg)
    content = bytes(content, "utf-8")
    self.send_content(content, HTTPStatus.NOT_FOUND)
```

-   Use `try`/`except` to handle errors in called methods

-   <a class="gl-ref" href="../../glossary/#gl:throw_low_catch_high" title="A widely-used pattern for managing exceptions whereby they are raised in many places at low levels of a program but caught in a few high-level places where corrective action can be taken." markdown="1">Throw low, catch high</a>

---

## Problems

-   Client can escape from our <a class="gl-ref" href="../../glossary/#gl:sandbox" title="A space where code can execute safely." markdown="1">sandbox</a>
    by asking for `http://localhost:8080/../../passwords.txt`

-   `send_content` always says it is returning HTML with `Content-Type`

-   Should use things like `image/png` for images

-   But we got <a class="gl-ref" href="../../glossary/#gl:character_encoding" title="A way to represent characters as bytes. Common examples include ASCII and UTF-8." markdown="1">character encoding</a> right

---

## Test Case

-   Want to write this

```py
def test_existing_path(fs):
    content_str = "actual"
    content_bytes = bytes(content_str, "utf-8")
    fs.create_file("/actual.txt", contents=content_str)
    handler = MockHandler("/actual.txt")
    handler.do_GET()
    assert handler.status == int(HTTPStatus.OK)
    assert handler.headers["Content-Type"] == \
        ["text/html; charset=utf-8"]
    assert handler.headers["Content-Length"] == \
        [str(len(content_bytes))]
    assert handler.wfile.getvalue() == content_bytes
```

---

## Combining Code

-   Use <a class="gl-ref" href="../../glossary/#gl:multiple_inheritance" title="Inheriting from two or more classes when creating a new class." markdown="1">multiple inheritance</a>

---

## Mock Request Handler

```py
from io import BytesIO

class MockRequestHandler:
    def __init__(self, path):
        self.path = path
        self.status = None
        self.headers = {}
        self.wfile = BytesIO()

def send_response(self, status):
        self.status = status

def send_header(self, key, value):
        if key not in self.headers:
            self.headers[key] = []
        self.headers[key].append(value)

def end_headers(self):
        pass
```

---

## Application Code

```py
class ApplicationRequestHandler:
    def do_GET(self):
        try:
            url_path = self.path.lstrip("/")
            full_path = Path.cwd().joinpath(url_path)
            if not full_path.exists():
                raise ServerException(f"'{self.path}' not found")
            elif full_path.is_file():
                self.handle_file(self.path, full_path)
            else:
                raise ServerException(f"Unknown object '{self.path}'")
        except Exception as msg:
            self.handle_error(msg)

# ...etc...
```

---

## Two Servers

```py
if __name__ == '__main__':
    class RequestHandler(
            BaseHTTPRequestHandler,
            ApplicationRequestHandler
    ):
        pass

serverAddress = ('', 8080)
    server = HTTPServer(serverAddress, RequestHandler)
    server.serve_forever()
```

```py
class MockHandler(
        MockRequestHandler,
        ApplicationRequestHandler
):
    pass
```

---

## Summary