release for 1.3.0

minor update, got ua blocking working and match statements in config
Updated README to reflect project updates
2025-07-22 16:35:39 +02:00 · 2025-07-08 15:02:50 +02:00 · 2025-07-08 14:37:27 +02:00 · 2025-06-15 13:33:52 +02:00 · 2025-06-15 13:33:14 +02:00 · 2025-06-15 13:32:19 +02:00
4 changed files with 159 additions and 90 deletions
--- a/README.md
+++ b/README.md
@@ -1,8 +1,5 @@
 # PyWebServer

-# PyWebServer is undergoing major security updates!
-# Please use commit [7ac160f625](https://git.novacow.ch/Nova/PyWebServer/src/commit/7ac160f6259e637c2337a672e56f105f4cdd2d2a) as the source for now!!
-
 ## GitHub
 The upstream of this project is on my own [Gitea instance](https://git.novacow.ch/Nova/PyWebServer/).  
 Because of that I'll mostly reply to issues and PRs there, you can submit issues and PRs on GitHub, but it might take longer before I read it.
@@ -19,14 +16,7 @@ Windows users, make sure you have installed Git, from there:
 git clone https://git.novacow.ch/Nova/PyWebServer.git
 Set-Location .\PyWebServer\
 ```
-From here, you should check from what directory you want to store the content in.  
-In this example, we'll use `./html/` (or `.\html\` for Windows users) from the perspective of the PyWebServer root dir.  
-To create this directory, do this:
-```bash
-mkdir ./html/
-```
-(This applies to both Windows and Linux)  
-Then, open `pywebsrv.conf` in your favorite text editor and change the `directory` key to the full path to the `./html/` you just created.  
+Then, open `pywebsrv.conf` in your favorite text editor and change the `directory` key to the full path where your files are stored.  
 After that, put your files in and run this:
 Linux:
 ```bash
@@ -41,17 +31,15 @@ py \path\to\pywebsrv.py
 ```

 ## SSL Support
-Currently PyWebServer warns about AutoCertGen not being installed. AutoCertGen currently is very unstable at the moment, and therefore is not available for download.  
 PyWebServer supports SSL/TLS for authentication via HTTPS. In the config file, you should enable the HTTPS port. After that you need to create the certificate.  
 Currently PyWebServer looks for the `cert.pem` and the `key.pem` files in the root directory of the installation.  
-PyWebServer comes with a test certificate, this certificate is self-signed, but doesn't have a matching issuer and subject. This is to prevent people from using it in production, even if they have disabled warnings of self-signed certificates.  

 ## HTTP support
 Currently PyWebServer only supports HTTP/1.1, this is very unlikely to change, as most of the modern web today still uses HTTP/1.1.  
 For methods PyWebServer only supports `GET`, this is being reworked though, check issue [#3](https://git.novacow.ch/Nova/PyWebServer/issues/3) for progress.

 ## Files support
-Unlike other small web servers, PyWebServer has full support for binary files being sent and received (once that logic is put in) over HTTP.
+Unlike other small web servers, PyWebServer has full support for binary files being sent and received (once that logic is put in) over HTTP(S).

 ## Support
 PyWebServer will follow a standard support scheme.
--- a/html/index.html
+++ b/html/index.html
@@ -0,0 +1,13 @@
+<!DOCTYPE html>
+<html>
+<head>
+    <title>Test page</title>
+</head>
+<body>
+    <h1>Hey there!</h1>
+    <h2>You're seeing this page because you haven't set up PyWebServer yet!</h2>
+    <h2>This page confirms that PyWebServer can read and serve files from your PC.</h2>
+    <h2>To make this go away, please edit the file `pywebsrv.conf` and edit the `directory` key to your directory of choice!</h2>
+    <p>Here you can simulate a 404 error: <a href="/uuh">Click me for a 404 error!</a></p>
+</body>
+</html>
--- a/pywebsrv.conf
+++ b/pywebsrv.conf
@@ -9,9 +9,6 @@ directory:<Enter directory here>
 # You can use FQDNs, IP-addresses and localhost,
 # Support for multiple hosts is coming.
 host:localhost
-# Ignores the host parameter (except for localhost) and allows everything.
-# DANGER! For obvious reasons this isn't recommended.
-allow-all:0
 # Enables HTTP support. (Only enables/disables the HTTP port.)
 http:1
 # Enables HTTPS support. (Only enables/disables the HTTPS port.)
@@ -26,6 +23,11 @@ disable-autocertgen:0
 # block-ip:0.0.0.0,1.1.1.1,2.2.2.2
 # If you wish to block User-Agents, this function is coming though.
 # block-ua:(NULL)
-# This function is deprecated, allows a connection with no Host header.
-# You should NEVER have to enable this! It can pose a risk to security!
-# allow-nohost:0
+
+# TEST: experimental non-defined keys go here:
+# keyfile key
+key-file:/home/nova/PyWebServer/key.pem
+# certfile keys
+cert-file:/home/nova/PyWebServer/cert.pem
+# allowed-methods, csv's
+allowed-methods:GET
--- a/pywebsrv.py
+++ b/pywebsrv.py
@@ -31,9 +31,12 @@ Simple to understand and mod codebase.
 All GNU GPL-3-or-above license. (Do with it what you want.)
 Library aswell as a standalone script:
    You can easily get access to other parts of the script if you need it.
+
+TODO: actually put normal comments in
 """

 import os
+import mimetypes
 import threading
 import ssl
 import socket
@@ -54,7 +57,7 @@ class FileHandler:
    DEFAULT_CONFIG = (
        "port:8080\nport-https:8443\nhttp:1"
        "\nhttps:0\ndirectory:{cwd}\nhost:localhost"
-        "allow-all:1\nallow-localhost:1"
+        "\nallow-localhost:1"
    )

    def __init__(self, base_dir=None):
@@ -71,23 +74,21 @@ class FileHandler:
        with open(self.config_path, "w") as f:
            f.write(self.DEFAULT_CONFIG.format(cwd=os.getcwd()))

-    def didnt_confirm(self):
-        os.remove(self.config_path)
-
    def read_file(self, file_path):
        if "../" in file_path:
-            return 403
+            return 403, None

        full_path = os.path.join(self.base_dir, file_path.lstrip("/"))
        if not os.path.isfile(full_path):
-            return 404
+            return 404, None

        try:
+            mimetype = mimetypes.guess_type(full_path)
            with open(full_path, "rb") as f:
-                return f.read()
+                return f.read(), mimetype
        except Exception as e:
            print(f"Error reading file {full_path}: {e}")
-            return 500
+            return 500, None

    def write_file(self, file_path, data):
        if "../" in file_path:
@@ -110,10 +111,11 @@ class FileHandler:
            "http",
            "https",
            "port-https",
-            "allow-all",
-            "allow-nohost",
            "allow-localhost",
            "disable-autocertgen",
+            "key-file",
+            "cert-file",
+            "block-ua"
        ]
        if option not in valid_options:
            return None
@@ -128,25 +130,35 @@ class FileHandler:
                key = key.lower()
                if key == option:
                    if option == "host":
-                        seperated_values = value.split(",", 0)
+                        seperated_values = value.split(",", -1)
                        return [value.lower() for value in seperated_values]
+                    if option == "block-ua":
+                        seperated_values = value.split(",", -1)
+                        host_to_match = []
+                        literal_blocks = []
+                        for val in seperated_values:
+                            if val.startswith("match(") and val.endswith(")"):
+                                idx = val.index("(")
+                                idx2 = val.index(")")
+                                ua_to_match = val[idx+1:idx2]
+                                host_to_match.append(ua_to_match)
+                            else:
+                                literal_blocks.append(val)
+                        return host_to_match, literal_blocks
                    if option == "port" or option == "port-https":
                        return int(value)
                    if (
                        option == "http"
                        or option == "https"
-                        or option == "allow-all"
                        or option == "allow-localhost"
                        or option == "disable-autocertgen"
-                        or option == "allow-nohost"
                    ):
                        return bool(int(value))
                    if option == "directory":
                        if value == "<Enter directory here>":
-                            print(
-                                "FATAL: You haven't set up PyWebServer! Please edit pywebsrv.conf!"
-                            )
-                            exit(1)
+                            return os.path.join(os.getcwd(), "html")
+                        if value.endswith("/"):
+                            value = value.rstrip("/")
                        return value
                    return value
        return None
@@ -161,10 +173,8 @@ class FileHandler:

 class RequestParser:
    def __init__(self):
-        self.allowed_methods_file = "allowedmethods.conf"
        self.file_handler = FileHandler()
        self.hosts = self.file_handler.read_config("host")
-        self.all_allowed = self.file_handler.read_config("allow-all")

    def parse_request_line(self, line):
        """Parses the HTTP request line."""
@@ -176,6 +186,16 @@ class RequestParser:
            path += "index.html"
        return method, path, version

+    def ua_blocker(self, ua):
+        """Parses and matches UA to block"""
+        match, literal = self.file_handler.read_config("block-ua")
+        if ua in literal:
+            return False
+        for _ua in match:
+            if _ua.lower() in ua.lower():
+                return False
+        return True
+
    def is_method_allowed(self, method):
        """
        Checks if the HTTP method is allowed.
@@ -186,9 +206,8 @@ class RequestParser:
        allowed_methods = ["GET"]
        # While the logic for PUT, DELETE, etc. is not added, we shouldn't
        # allow for it to attempt it.
-        # if os.path.isfile(self.allowed_methods_file):
-        #     with open(self.allowed_methods_file, "r") as f:
-        #         allowed_methods = [line.strip() for line in f]
+        # Prepatched for new update.
+        # allowed_methods = self.file_handler.read_config("allowed-methods")
        return method in allowed_methods

    def host_parser(self, host):
@@ -196,17 +215,18 @@ class RequestParser:
        Parses the host and makes sure it's allowed in
        Mfw im in an ugly code writing contest and my opponent is nova while writing a side project
        """
-        host = str(host)
+        host = f"{host}"
        if ":" in host:
            host = host.split(":", 1)[0]
        host = host.lstrip()
+        host = host.rstrip()
        if (
            host == "localhost" or host == "127.0.0.1"
        ) and self.file_handler.read_config("allow-localhost"):
            return True
-        if host not in self.hosts and self.all_allowed is False:
+        if host not in self.hosts:
            return False
-        elif host not in self.hosts and self.all_allowed is True:
+        else:
            return True


@@ -224,6 +244,13 @@ class WebServer:

        # me when no certificate and key file
        if not os.path.exists(self.cert_file) or not os.path.exists(self.key_file):
+            if not os.path.exists(self.cert_file) and not os.path.exists(self.key_file):
+                pass
+            # maybe warn users we purge their key/cert files? xdd
+            elif not os.path.exists(self.cert_file):
+                os.remove(self.key_file)
+            elif not os.path.exists(self.key_file):
+                os.remove(self.cert_file)
            print("WARN: No HTTPS certificate was found!")
            if self.file_handler.read_config("disable-autocertgen") is True:
                print("WARN: AutoCertGen is disabled, ignoring...")
@@ -235,11 +262,8 @@ class WebServer:
                else:
                    self.skip_ssl = True

-        # TODO: change this to something like oh no you fucked up, go fix idiot
        self.no_host_req_response = (
-            "Connecting via this host is disallowed\r\n"
-            "You may also be using a very old browser!\r\n"
-            "Ask the owner of this website to set allow-all to 1!"
+            "This host cannot be reached without sending a `Host` header."
        )

        # ipv6 when????/??//?????//?
@@ -263,17 +287,17 @@ class WebServer:

        self.http_404_html = (
            "<html><head><title>HTTP 404 - PyWebServer</title></head>"
-            "<body><center><h1>HTTP 404 - Not Found!</h1><p>Running PyWebServer/1.1+u2</p>"
+            "<body><center><h1>HTTP 404 - Not Found!</h1><p>Running PyWebServer/1.2.1</p>"
            "</center></body></html>"
        )
        self.http_403_html = (
            "<html><head><title>HTTP 403 - PyWebServer</title></head>"
-            "<body><center><h1>HTTP 403 - Forbidden</h1><p>Running PyWebServer/1.1+u2</p>"
+            "<body><center><h1>HTTP 403 - Forbidden</h1><p>Running PyWebServer/1.2.1</p>"
            "</center></body></html>"
        )
        self.http_405_html = (
            "<html><head><title>HTTP 405 - PyWebServer</title></head>"
-            "<body><center><h1>HTTP 405 - Method not allowed</h1><p>Running PyWebServer/1.1+u2</p>"
+            "<body><center><h1>HTTP 405 - Method not allowed</h1><p>Running PyWebServer/1.2.1</p>"
            "</center></body></html>"
        )

@@ -287,14 +311,13 @@ class WebServer:
        https_thread = threading.Thread(target=self.start_https, daemon=True)

        if https is True:
+            if self.skip_ssl is True:
+                print("WARN: You have enabled HTTPS without SSL!!")
+                yn = input("Is this intended behaviour? [y/N] ")
            https_thread.start()
        if http is True:
            http_thread.start()

-        print(
-            f"Server running:\n - HTTP on port {self.http_port}\n - HTTPS on port {self.https_port}"
-        )
-
        http_thread.join()
        https_thread.join()

@@ -304,7 +327,6 @@ class WebServer:
        while self.running:
            try:
                conn, addr = self.http_socket.accept()
-                print(f"HTTP connection received from {addr}")
                self.handle_connection(conn, addr)
            except Exception as e:
                print(f"HTTP error: {e}")
@@ -317,7 +339,6 @@ class WebServer:
        while self.running:
            try:
                conn, addr = self.https_socket.accept()
-                print(f"HTTPS connection received from {addr}")
                self.handle_connection(conn, addr)
            except Exception as e:
                print(
@@ -330,7 +351,12 @@ class WebServer:
        try:
            data = conn.recv(512)
            request = data.decode(errors="ignore")
-            response = self.handle_request(request, addr)
+            if not data:
+                response = self.build_response(400, "Bad Request")  # user did fucky-wucky
+            elif len(data) > 8192:
+                response = self.build_response(413, "Request too long")
+            else:
+                response = self.handle_request(request, addr)

            if isinstance(response, str):
                response = response.encode()
@@ -342,9 +368,7 @@ class WebServer:
            conn.close()

    def handle_request(self, data, addr):
-        if not data:
-            return self.build_response(400, "Bad Request")  # user did fucky-wucky
-
+        print(f"data: {data}")
        request_line = data.splitlines()[0]

        # Extract host from headers, never works though
@@ -358,22 +382,42 @@ class WebServer:
                    )
                break
        else:
-            if (
-                self.file_handler.read_config("allow-nohost") is True
-            ):  # no host is stupid
-                pass
            return self.build_response(
-                403, self.no_host_req_response.encode()
-            )  # the default (i hope to god)
+                400, self.no_host_req_response.encode()
+            )
+
+        for line in data.splitlines():
+            if "User-Agent" in line:
+                ua = line.split(":", 1)[1].strip()
+                allowed = self.parser.ua_blocker(ua)
+                if not allowed:
+                    return self.build_response(
+                        403, "This UA has been blocked by the owner of this site."
+                    )
+                break
+        else:
+            return self.build_response(
+                400, "You cannot connect without a User-Agent."
+            )

        method, path, version = self.parser.parse_request_line(request_line)

-        if not all([method, path, version]) or not self.parser.is_method_allowed(
+        if not all([method, path, version]):
+            return self.build_response(400, "Bad Request")
+
+        # Figure out a better way to reload config
+        if path == "/?pywebsrv_reload_conf=1":
+            print("Got reload command! Reloading configuration...")
+            self.file_handler = FileHandler()
+            self.parser = RequestParser()
+            return self.build_response(302, "")
+
+        if not self.parser.is_method_allowed(
            method
        ):
            return self.build_response(405, self.http_405_html)

-        file_content = self.file_handler.read_file(path)
+        file_content, mimetype = self.file_handler.read_file(path)

        if file_content == 403:
            print("WARN: Directory traversal attack prevented.")  # look ma, security!!
@@ -389,72 +433,94 @@ class WebServer:

        # A really crude implementation of binary files. Later in 2.0 I'll actually
        # make this useful.
-        if path.endswith((".mp3", ".png", ".jpg", ".jpeg", ".gif")):
-            return self.build_binary_response(200, file_content, path)
+        mimetype = mimetype[0]
+        if mimetype is None:
+            # We have to assume it's binary.
+            return self.build_binary_response(200, file_content, "application/octet-stream")
+        if "text/" not in mimetype:
+            return self.build_binary_response(200, file_content, mimetype)

        return self.build_response(200, file_content)

    @staticmethod
-    def build_binary_response(status_code, binary_data, filename):
+    def build_binary_response(status_code, binary_data, content_type):
        """Handles binary files like MP3s."""
        messages = {
            200: "OK",
            403: "Forbidden",
            404: "Not Found",
            405: "Method Not Allowed",
-            500: "Internal Server Error",
+            500: "Internal Server Error"
        }
        status_message = messages.get(status_code)
-
-        # In the spirit of keeping stuff small, we'll just guess and see.
-        content_type = "application/octet-stream"
-        if filename.endswith(".mp3"):
-            content_type = "audio/mpeg"
-        elif filename.endswith(".png"):
-            content_type = "image/png"
-        elif filename.endswith(".jpg") or filename.endswith(".jpeg"):
-            content_type = "image/jpeg"
-        elif filename.endswith(".gif"):
-            content_type = "image/gif"
-
        headers = (
            f"HTTP/1.1 {status_code} {status_message}\r\n"
-            f"Server: PyWebServer/1.1+u2\r\n"
+            f"Server: PyWebServer/1.2.1\r\n"
            f"Content-Type: {content_type}\r\n"
            f"Content-Length: {len(binary_data)}\r\n"
            f"Connection: close\r\n\r\n"
            # Connection close is done because it is way easier to implement.
            # It's not like this program will see production use anyway.
+            # Tbh when i'll implement HTTP2
        )
        return headers.encode() + binary_data

-    @staticmethod
-    def build_response(status_code, body):
+    def build_response(self, status_code, body):
        """
        For textfiles we'll not have to guess MIME-types, though the other function
        build_binary_response will be merged in here anyway.
        """
        messages = {
            200: "OK",
+            204: "No Content",
+            302: "Found",
            304: "Not Modified",  # TODO KEKL
            400: "Bad Request",
            403: "Forbidden",
            404: "Not Found",
            405: "Method Not Allowed",
+            413: "Payload Too Large",
            500: "Internal Server Error",
+            635: "Go Away"
        }
        status_message = messages.get(status_code)

        if isinstance(body, str):
            body = body.encode()

+        # TODO: dont encode yet, and i encode. awesome comments here.
+        # Don't encode yet, if 302 status code we have to include location.
        headers = (
            f"HTTP/1.1 {status_code} {status_message}\r\n"
-            f"Server: PyWebServer/1.1+u2\r\n"
+            f"Server: PyWebServer/1.2.1\r\n"
            f"Content-Length: {len(body)}\r\n"
            f"Connection: close\r\n\r\n"
        ).encode()

+        if status_code == 302:
+            # 302 currently only happens when the reload is triggered.
+            # Why not 307, Moved Permanently? Because browsers will cache the
+            # response and not send the reload command.
+            host = self.file_handler.read_config("host")[0]
+            port = self.file_handler.read_config("port-https") or self.file_handler.read_config("port")
+            if port != 80 and port != 443:
+                if port == 8443:
+                    host = f"https://{host}:{port}/"
+                else:
+                    host = f"http://{host}:{port}/"
+            else:
+                if port == 443:
+                    host = f"https://{host}/"
+                else:
+                    host = f"http://{host}/"
+            headers = (
+                f"HTTP/1.1 {status_code} {status_message}\r\n"
+                f"Location: {host}\r\n"
+                f"Server: PyWebServer/1.2.1\r\n"
+                f"Content-Length: {len(body)}\r\n"
+                f"Connection: close\r\n\r\n"
+            ).encode()
+
        return headers + body

    def shutdown(self, signum, frame):
Author	SHA1	Message	Date
Nova	a36141edd0	release for 1.3.0 minor update, got ua blocking working and match statements in config	2025-07-22 16:35:39 +02:00
Stan	0b57578cc4	Updated README to reflect project updates	2025-07-08 15:02:50 +02:00
Stan	d6fdc5c63d	v1.2.1: small bugfixes and welcome page.	2025-07-08 14:37:27 +02:00
Stan	590abbf649	Uuuh?	2025-06-15 13:33:52 +02:00
Stan	87a2505395	Merge branch 'main' of https://git.novacow.ch/Nova/PyWebServer	2025-06-15 13:33:14 +02:00
Stan	118a342e9d	Update to v1.2, crude fix for #6 and for #5	2025-06-15 13:32:19 +02:00
Nova	9cd0528ab3	Update README.md	2025-05-04 14:54:51 +02:00