Update README.md

minor update, got ua blocking working and match statements in config

README.md

@@ -1,10 +1,56 @@
 # PyWebServer
 
+## Current state
+Currently I'm in the middle of bringing major improvements to PyWebServer, like a significantly better config
+automatic HTTPS certificate generation using Let's Encrypt, HTTP `PUT`, `POST` and `DELETE` support,
+HTTP2 support and auto-indexing.  
+This all will be part of major release 2, version 2.0. These improvements will partially make their way into 1.x
+versions, from 1.4 onwards, mostly as testing ground and compatibility with 2.0 reasons. Most will probably be dead code
+until 2.0 properly comes out. Until then, parts of the code may be pretty cluttered. I'll probably enable you to test some feautres
+like the new config, but code quality is currently not a main priority.
+
 ## GitHub
-The upstream of this project is on my own [Gitea instance](https://git.novacow.ch/Nova/PyWebServer/).  
+The host of this project is on my own [Gitea instance](https://git.novacow.ch/Nova/PyWebServer/).  
 Because of that I'll mostly reply to issues and PRs there, you can submit issues and PRs on GitHub, but it might take longer before I read it.
 
 ## Installing
+### The little tiny easier route
+Installing and running PyWebServer is very simple.  
+First off, download the latest release from the 'Releases' tab, choose the Zip variant if unsure.  
+When it's done downloading, unpack the files in a directory of choice, for the purpose of this README, 
+I've chosen `./pywebserver/` (for Windows: `.\pywebserver\`).  
+From there, open up your favorite text editor and open the file `pywebsrv.conf` in the directory you unpacked PyWebServer.  
+In there, you should see this somewhere:
+```
+# Here you choose what directory PyWebServer looks in for files.
+directory:<Enter directory here>
+```
+After the colon, enter your directory where you have your website stored.  
+After that, make sure you have installed Python. Here's how you can install Python:  
+Linux:
+```bash
+sudo apt install python3  # Debian / Ubuntu
+sudo dnf install python3  # Fedora / Nobara
+sudo pacman -S python3  # Arch and derivatives.
+```
+macOS:
+```bash
+brew install python3
+```
+Windows:
+```powershell
+# You can change the `3.12` with whatever version you want/need.
+winget install -e --id Python.Python.3.12 --scope machine
+```
+Then, in the terminal window you have open, go to the directory you unpacked PyWebServer and type this:
+```
+python3 ./pywebsrv.py
+# For Windows users, if the above command doesn't work, try this:
+py ./pywebsrv.py
+```
+And there you go! You've now set up PyWebServer!
+
+### The little tiny harder route
 Installing and running PyWebServer is very simple.  
 Assuming you're running Linux:
 ```bash
@@ -48,4 +94,4 @@ For every 1.x version there will be support until 2 newer versions come out.
 So that means that 1.0 will still be supported when 1.1 comes out, but no longer be supported when 1.2 comes out.
 ### 2.x
 I am planning on releasing a 2.x version with will have a lot more advanced features, like nginx's server block emulation amongst other things.
-When 2.0 will come out, the last version of 1.x will be supported for a while longer, but no new features will be added.
+When 2.0 will come out, 1.x will be developed further, but 2.0 will be the main focus.

pywebsrv.py

@@ -12,6 +12,8 @@ License:
     Contact:
         E-mail: nova@novacow.ch
 
+NOTE: Once 2.0 is released, PyWebServer will become the Amethyst Web Server
+
 This is PyWebServer, an ultra minimalist webserver, meant to still have
 a lot standard webserver features. A comprehensive list is below:
 Features:
@@ -40,39 +42,35 @@ import mimetypes
 import threading
 import ssl
 import socket
+import re
 import signal
 import sys
 
 try:
     from certgen import AutoCertGen
 except ImportError:
-    print(
-        "WARN: You need the AutoCertGen plugin! Please install it from\n"
-        "https://git.novacow.ch/Nova/AutoCertGen/"
-    )
+    # just do nothing, it's not working anyway.
+    # print(
+    #     "WARN: You need the AutoCertGen plugin! Please install it from\n"
+    #     "https://git.novacow.ch/Nova/AutoCertGen/"
+    # )
+    pass
 
 
 class FileHandler:
     CONFIG_FILE = "pywebsrv.conf"
-    DEFAULT_CONFIG = (
-        "port:8080\nport-https:8443\nhttp:1"
-        "\nhttps:0\ndirectory:{cwd}\nhost:localhost"
-        "\nallow-localhost:1"
-    )
 
     def __init__(self, base_dir=None):
         self.config_path = os.path.join(os.getcwd(), self.CONFIG_FILE)
         self.base_dir = self.read_config("directory")
-
-    def check_first_run(self):
-        if not os.path.isfile(self.config_path):
-            self.on_first_run()
-            return True
-        return False
-
-    def on_first_run(self):
-        with open(self.config_path, "w") as f:
-            f.write(self.DEFAULT_CONFIG.format(cwd=os.getcwd()))
+        self.cached_conf = None
+        if not os.path.exists(self.config_path):
+            print(
+                "The pywebsrv.conf file needs to be in the same directory "
+                "as pywebsrv.py! Get the default config file from:\n"
+                "https://git.novacow.ch/Nova/PyWebServer/raw/branch/main/pywebsrv.conf"
+            )
+            exit(1)
 
     def read_file(self, file_path):
         if "../" in file_path:
@@ -114,7 +112,8 @@ class FileHandler:
             "allow-localhost",
             "disable-autocertgen",
             "key-file",
-            "cert-file"
+            "cert-file",
+            "block-ua"
         ]
         if option not in valid_options:
             return None
@@ -131,6 +130,19 @@ class FileHandler:
                     if option == "host":
                         seperated_values = value.split(",", -1)
                         return [value.lower() for value in seperated_values]
+                    if option == "block-ua":
+                        seperated_values = value.split(",", -1)
+                        host_to_match = []
+                        literal_blocks = []
+                        for val in seperated_values:
+                            if val.startswith("match(") and val.endswith(")"):
+                                idx = val.index("(")
+                                idx2 = val.index(")")
+                                ua_to_match = val[idx+1:idx2]
+                                host_to_match.append(ua_to_match)
+                            else:
+                                literal_blocks.append(val)
+                        return host_to_match, literal_blocks
                     if option == "port" or option == "port-https":
                         return int(value)
                     if (
@@ -149,9 +161,55 @@ class FileHandler:
                     return value
         return None
 
+    def read_new_config(self, option, host=None):
+        """
+        Reads the configuration file and returns a dict
+        """
+        if self.cached_conf is None:
+            with open(self.config_path, "r", encoding="utf-8") as fh:
+                text = fh.read()
+
+            blocks = re.findall(
+                r'^(host\s+(\S+)|globals)\s*\{([^}]*)\}', text, re.MULTILINE
+            )
+            parsed = {}
+            host_list = []
+            for tag, hostname, body in blocks:
+                section = hostname if hostname else "globals"
+                if hostname:
+                    host_list.append(hostname)
+                kv = {}
+                for line in body.splitlines():
+                    line = line.strip()
+                    if not line or ":" not in line or line.starswith("#"):
+                        continue
+
+                    key, rest = line.split(":", 1)
+                    key = key.strip()
+                    rest = rest.strip()
+
+                    # Split comma-separated values (e.g. GET,PUT)
+                    if "," in rest:
+                        kv[key] = [item.strip() for item in rest.split(",")]
+                    else:
+                        kv[key] = rest
+                parsed[section] = kv
+                parsed["globals"]["hosts"] = host_list
+                self.cached_conf = parsed
+            else:
+                parsed = self.cached_conf
+            if option == "host":
+                try:
+                    return host_list
+                except Exception:
+                    return parsed["globals"]["hosts"]
+            section = parsed.get(host or "globals", {})
+            return section.get(option)
+
     def autocert(self):
         """
         Generate some self-signed certificates using AutoCertGen
+        TODO: doesn't work, need to fix. probably add `./` to $PATH
         """
         autocert = AutoCertGen()
         autocert.gen_cert()
@@ -172,6 +230,16 @@ class RequestParser:
             path += "index.html"
         return method, path, version
 
+    def ua_blocker(self, ua):
+        """Parses and matches UA to block"""
+        match, literal = self.file_handler.read_config("block-ua")
+        if ua in literal:
+            return False
+        for _ua in match:
+            if _ua.lower() in ua.lower():
+                return False
+        return True
+
     def is_method_allowed(self, method):
         """
         Checks if the HTTP method is allowed.
@@ -205,6 +273,12 @@ class RequestParser:
         else:
             return True
 
+#
+# class ProxyServer:
+#     def __init__(
+#         self,
+#     ):
+
 
 class WebServer:
     def __init__(
@@ -303,7 +377,6 @@ class WebServer:
         while self.running:
             try:
                 conn, addr = self.http_socket.accept()
-                print(f"HTTP connection received from {addr}")
                 self.handle_connection(conn, addr)
             except Exception as e:
                 print(f"HTTP error: {e}")
@@ -316,7 +389,6 @@ class WebServer:
         while self.running:
             try:
                 conn, addr = self.https_socket.accept()
-                print(f"HTTPS connection received from {addr}")
                 self.handle_connection(conn, addr)
             except Exception as e:
                 print(
@@ -329,6 +401,11 @@ class WebServer:
         try:
             data = conn.recv(512)
             request = data.decode(errors="ignore")
+            if not data:
+                response = self.build_response(400, "Bad Request")  # user did fucky-wucky
+            elif len(data) > 8192:
+                response = self.build_response(413, "Request too long")
+            else:
                 response = self.handle_request(request, addr)
 
             if isinstance(response, str):
@@ -341,12 +418,7 @@ class WebServer:
             conn.close()
 
     def handle_request(self, data, addr):
-        print(f"len data: {len(data)}")
-        if not data:
-            return self.build_response(400, "Bad Request")  # user did fucky-wucky
-        if len(data) > 8192:
-            return self.build_response(413, "Request too long")
-
+        print(f"data: {data}")
         request_line = data.splitlines()[0]
 
         # Extract host from headers, never works though
@@ -364,17 +436,32 @@ class WebServer:
                 400, self.no_host_req_response.encode()
             )
 
+        for line in data.splitlines():
+            if "User-Agent" in line:
+                ua = line.split(":", 1)[1].strip()
+                allowed = self.parser.ua_blocker(ua)
+                if not allowed:
+                    return self.build_response(
+                        403, "This UA has been blocked by the owner of this site."
+                    )
+                break
+        else:
+            return self.build_response(
+                400, "You cannot connect without a User-Agent."
+            )
+
         method, path, version = self.parser.parse_request_line(request_line)
 
+        if not all([method, path, version]):
+            return self.build_response(400, "Bad Request")
+
         # Figure out a better way to reload config
         if path == "/?pywebsrv_reload_conf=1":
             print("Got reload command! Reloading configuration...")
-            self.file_handler.base_dir = self.file_handler.read_config("directory")
+            self.file_handler = FileHandler()
+            self.parser = RequestParser()
             return self.build_response(302, "")
 
-        if not all([method, path, version]):
-            return self.build_response(400, "Bad Request")
-
         if not self.parser.is_method_allowed(
             method
         ):
@@ -397,6 +484,9 @@ class WebServer:
         # A really crude implementation of binary files. Later in 2.0 I'll actually
         # make this useful.
         mimetype = mimetype[0]
+        if mimetype is None:
+            # We have to assume it's binary.
+            return self.build_binary_response(200, file_content, "application/octet-stream")
         if "text/" not in mimetype:
             return self.build_binary_response(200, file_content, mimetype)
 
@@ -410,17 +500,18 @@ class WebServer:
             403: "Forbidden",
             404: "Not Found",
             405: "Method Not Allowed",
-            500: "Internal Server Error",
+            500: "Internal Server Error"
         }
         status_message = messages.get(status_code)
         headers = (
             f"HTTP/1.1 {status_code} {status_message}\r\n"
-            f"Server: PyWebServer/1.2.1\r\n"
+            f"Server: PyWebServer/1.4\r\n"
             f"Content-Type: {content_type}\r\n"
             f"Content-Length: {len(binary_data)}\r\n"
             f"Connection: close\r\n\r\n"
             # Connection close is done because it is way easier to implement.
             # It's not like this program will see production use anyway.
+            # Tbh when i'll implement HTTP2
         )
         return headers.encode() + binary_data
 
@@ -440,7 +531,7 @@ class WebServer:
             405: "Method Not Allowed",
             413: "Payload Too Large",
             500: "Internal Server Error",
-            635: "Go Away",
+            635: "Go Away"
         }
         status_message = messages.get(status_code)
 
@@ -451,7 +542,7 @@ class WebServer:
         # Don't encode yet, if 302 status code we have to include location.
         headers = (
             f"HTTP/1.1 {status_code} {status_message}\r\n"
-            f"Server: PyWebServer/1.2.1\r\n"
+            f"Server: PyWebServer/1.4\r\n"
             f"Content-Length: {len(body)}\r\n"
             f"Connection: close\r\n\r\n"
         ).encode()