Build 0001, first Amethyst version that starts!

Partial new config functionality.

README.md

@@ -1,51 +1,9 @@
-# PyWebServer
+# Amethyst Web Server
 
-## GitHub
-The upstream of this project is on my own [Gitea instance](https://git.novacow.ch/Nova/PyWebServer/).  
-Because of that I'll mostly reply to issues and PRs there, you can submit issues and PRs on GitHub, but it might take longer before I read it.
+## A word of warning!
+Currently Amethyst is in very early alpha stage, a lot of things will be broken, names won't be correct,
+promised features missing, but I'm very much working on it live!  
+Every save I do increments the build number by 1, I won't publish all of them, but most of them will be published.  
+Once a milestone is hit (e.g. a new feature fully implemented), I'll publish a release!
 
-## Installing
-Installing and running PyWebServer is very simple.  
-Assuming you're running Linux:
-```bash
-git clone https://git.novacow.ch/Nova/PyWebServer.git
-cd ./PyWebServer/
-```
-Windows users, make sure you have installed Git, from there:
-```powershell
-git clone https://git.novacow.ch/Nova/PyWebServer.git
-Set-Location .\PyWebServer\
-```
-Then, open `pywebsrv.conf` in your favorite text editor and change the `directory` key to the full path where your files are stored.  
-After that, put your files in and run this:
-Linux:
-```bash
-python3 /path/to/pywebsrv.py
-```
-Windows:
-```powershell
-# If you have installed Python via the Microsoft Store:
-python3 \path\to\pywebsrv.py
-# Via the python.org website:
-py \path\to\pywebsrv.py
-```
-
-## SSL Support
-PyWebServer supports SSL/TLS for authentication via HTTPS. In the config file, you should enable the HTTPS port. After that you need to create the certificate.  
-Currently PyWebServer looks for the `cert.pem` and the `key.pem` files in the root directory of the installation.  
-
-## HTTP support
-Currently PyWebServer only supports HTTP/1.1, this is very unlikely to change, as most of the modern web today still uses HTTP/1.1.  
-For methods PyWebServer only supports `GET`, this is being reworked though, check issue [#3](https://git.novacow.ch/Nova/PyWebServer/issues/3) for progress.
-
-## Files support
-Unlike other small web servers, PyWebServer has full support for binary files being sent and received (once that logic is put in) over HTTP(S).
-
-## Support
-PyWebServer will follow a standard support scheme.
-### 1.x
-For every 1.x version there will be support until 2 newer versions come out.
-So that means that 1.0 will still be supported when 1.1 comes out, but no longer be supported when 1.2 comes out.
-### 2.x
-I am planning on releasing a 2.x version with will have a lot more advanced features, like nginx's server block emulation amongst other things.
-When 2.0 will come out, the last version of 1.x will be supported for a while longer, but no new features will be added.
+## Currently W.I.P. Check back later!

new_conf.conf

@@ -0,0 +1,30 @@
+# WARNING: This is an alpha spec of NSCL 2.0!!
+
+host example.com {
+    directory:/home/nova/Downloads/test/html
+    allowed-methods:GET
+    block-ip:match-ip("192.168",2)
+    block-ua:match("Discordbot",0),match("Google",0)
+}
+
+host cdn.example.com {
+    directory:/home/nova/Downloads/test/cdn
+    allowed-methods:GET,PUT
+    block-ip:10.1.100.2
+    block-ua:match("Discordbot",0)
+}
+
+host modem.example.com {
+    proxy:192.168.2.254
+    key-file:/home/nova/Downloads/test/proxykey.pem
+    cert-file:/home/nova/Downloads/test/proxycert.pem
+}
+
+globals {
+    http:1
+    https:1
+    port:8080
+    https-port:8443
+    global-key:/home/nova/Downloads/test/key.pem
+    global-cert:/home/nova/Downloads/test/cert.pem
+}

pywebsrv.py

@@ -12,6 +12,8 @@ License:
     Contact:
         E-mail: nova@novacow.ch
 
+NOTE: Once 2.0 is released, PyWebServer will become the Amethyst Web Server
+
 This is PyWebServer, an ultra minimalist webserver, meant to still have
 a lot standard webserver features. A comprehensive list is below:
 Features:
@@ -40,39 +42,39 @@ import mimetypes
 import threading
 import ssl
 import socket
+import re
 import signal
 import sys
 
 try:
     from certgen import AutoCertGen
 except ImportError:
-    print(
-        "WARN: You need the AutoCertGen plugin! Please install it from\n"
-        "https://git.novacow.ch/Nova/AutoCertGen/"
-    )
+    # just do nothing, it's not working anyway.
+    # print(
+    #     "WARN: You need the AutoCertGen plugin! Please install it from\n"
+    #     "https://git.novacow.ch/Nova/AutoCertGen/"
+    # )
+    pass
 
+AMETHYST_BUILD_NUMBER = "0001"
+AMETHYST_REPO = "https://git.novacow.ch/Nova/PyWebServer/"
 
 class FileHandler:
     CONFIG_FILE = "pywebsrv.conf"
-    DEFAULT_CONFIG = (
-        "port:8080\nport-https:8443\nhttp:1"
-        "\nhttps:0\ndirectory:{cwd}\nhost:localhost"
-        "\nallow-localhost:1"
-    )
+    new_conf = "new_conf.conf"
 
     def __init__(self, base_dir=None):
         self.config_path = os.path.join(os.getcwd(), self.CONFIG_FILE)
+        self.new_conf = os.path.join(os.getcwd(), self.new_conf)
         self.base_dir = self.read_config("directory")
-
-    def check_first_run(self):
-        if not os.path.isfile(self.config_path):
-            self.on_first_run()
-            return True
-        return False
-
-    def on_first_run(self):
-        with open(self.config_path, "w") as f:
-            f.write(self.DEFAULT_CONFIG.format(cwd=os.getcwd()))
+        self.cached_conf = None
+        if not os.path.exists(self.config_path):
+            print(
+                "The pywebsrv.conf file needs to be in the same directory "
+                "as pywebsrv.py! Get the default config file from:\n"
+                "https://git.novacow.ch/Nova/PyWebServer/raw/branch/main/pywebsrv.conf"
+            )
+            exit(1)
 
     def read_file(self, file_path):
         if "../" in file_path:
@@ -114,7 +116,8 @@ class FileHandler:
             "allow-localhost",
             "disable-autocertgen",
             "key-file",
-            "cert-file"
+            "cert-file",
+            "block-ua"
         ]
         if option not in valid_options:
             return None
@@ -131,6 +134,19 @@ class FileHandler:
                     if option == "host":
                         seperated_values = value.split(",", -1)
                         return [value.lower() for value in seperated_values]
+                    if option == "block-ua":
+                        seperated_values = value.split(",", -1)
+                        host_to_match = []
+                        literal_blocks = []
+                        for val in seperated_values:
+                            if val.startswith("match(") and val.endswith(")"):
+                                idx = val.index("(")
+                                idx2 = val.index(")")
+                                ua_to_match = val[idx+1:idx2]
+                                host_to_match.append(ua_to_match)
+                            else:
+                                literal_blocks.append(val)
+                        return host_to_match, literal_blocks
                     if option == "port" or option == "port-https":
                         return int(value)
                     if (
@@ -149,9 +165,56 @@ class FileHandler:
                     return value
         return None
 
+    def read_new_config(self, option, host=None):
+        """
+        Reads the configuration file and returns a dict
+        """
+        if self.cached_conf is None:
+            with open(self.new_conf, "r", encoding="utf-8") as fh:
+                text = fh.read()
+
+            blocks = re.findall(
+                r'^(host\s+(\S+)|globals)\s*\{([^}]*)\}', text, re.MULTILINE
+            )
+            parsed = {}
+            host_list = []
+            print(f"Blocks: {blocks}")
+            for tag, hostname, body in blocks:
+                section = hostname if hostname else "globals"
+                if hostname:
+                    host_list.append(hostname)
+                kv = {}
+                for line in body.splitlines():
+                    line = line.strip()
+                    if not line or ":" not in line or line.startswith("#"):
+                        continue
+
+                    key, rest = line.split(":", 1)
+                    key = key.strip()
+                    rest = rest.strip()
+
+                    # Split comma-separated values (e.g. GET,PUT)
+                    if "," in rest:
+                        kv[key] = [item.strip() for item in rest.split(",")]
+                    else:
+                        kv[key] = rest
+            parsed[section] = kv
+            parsed["globals"]["hosts"] = host_list
+            self.cached_conf = parsed
+        else:
+            parsed = self.cached_conf
+        if option == "host":
+            try:
+                return host_list
+            except Exception:
+                return parsed["globals"]["hosts"]
+        section = parsed.get(host or "globals", {})
+        return section.get(option)
+
     def autocert(self):
         """
         Generate some self-signed certificates using AutoCertGen
+        TODO: doesn't work, need to fix. probably add `./` to $PATH
         """
         autocert = AutoCertGen()
         autocert.gen_cert()
@@ -160,7 +223,8 @@ class FileHandler:
 class RequestParser:
     def __init__(self):
         self.file_handler = FileHandler()
-        self.hosts = self.file_handler.read_config("host")
+        self.hosts = self.file_handler.read_new_config("host")
+        print(f"Hosts: {self.hosts}")
 
     def parse_request_line(self, line):
         """Parses the HTTP request line."""
@@ -172,6 +236,17 @@ class RequestParser:
             path += "index.html"
         return method, path, version
 
+    def ua_blocker(self, ua, host=None):
+        """Parses and matches UA to block"""
+        del host
+        match, literal = self.file_handler.read_config("block-ua")
+        if ua in literal:
+            return False
+        for _ua in match:
+            if _ua.lower() in ua.lower():
+                return False
+        return True
+
     def is_method_allowed(self, method):
         """
         Checks if the HTTP method is allowed.
@@ -192,6 +267,7 @@ class RequestParser:
         Mfw im in an ugly code writing contest and my opponent is nova while writing a side project
         """
         host = f"{host}"
+        print(f"hosts: {self.hosts}, host: {host}")
         if ":" in host:
             host = host.split(":", 1)[0]
         host = host.lstrip()
@@ -205,13 +281,19 @@ class RequestParser:
         else:
             return True
 
+#
+# class ProxyServer:
+#     def __init__(
+#         self,
+#     ):
+
 
 class WebServer:
     def __init__(
         self, http_port=8080, https_port=8443, cert_file="cert.pem", key_file="key.pem"
     ):
-        self.http_port = http_port
-        self.https_port = https_port
+        self.http_port = int(http_port)
+        self.https_port = int(https_port)
         self.cert_file = cert_file
         self.key_file = key_file
         self.file_handler = FileHandler()
@@ -263,17 +345,17 @@ class WebServer:
 
         self.http_404_html = (
             "<html><head><title>HTTP 404 - PyWebServer</title></head>"
-            "<body><center><h1>HTTP 404 - Not Found!</h1><p>Running PyWebServer/1.2.1</p>"
+            f"<body><center><h1>HTTP 404 - Not Found!</h1><p>Running PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}</p>"
             "</center></body></html>"
         )
         self.http_403_html = (
             "<html><head><title>HTTP 403 - PyWebServer</title></head>"
-            "<body><center><h1>HTTP 403 - Forbidden</h1><p>Running PyWebServer/1.2.1</p>"
+            f"<body><center><h1>HTTP 403 - Forbidden</h1><p>Running PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}</p>"
             "</center></body></html>"
         )
         self.http_405_html = (
             "<html><head><title>HTTP 405 - PyWebServer</title></head>"
-            "<body><center><h1>HTTP 405 - Method not allowed</h1><p>Running PyWebServer/1.2.1</p>"
+            f"<body><center><h1>HTTP 405 - Method not allowed</h1><p>Running PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}</p>"
             "</center></body></html>"
         )
 
@@ -303,7 +385,6 @@ class WebServer:
         while self.running:
             try:
                 conn, addr = self.http_socket.accept()
-                print(f"HTTP connection received from {addr}")
                 self.handle_connection(conn, addr)
             except Exception as e:
                 print(f"HTTP error: {e}")
@@ -316,7 +397,6 @@ class WebServer:
         while self.running:
             try:
                 conn, addr = self.https_socket.accept()
-                print(f"HTTPS connection received from {addr}")
                 self.handle_connection(conn, addr)
             except Exception as e:
                 print(
@@ -329,7 +409,12 @@ class WebServer:
         try:
             data = conn.recv(512)
             request = data.decode(errors="ignore")
-            response = self.handle_request(request, addr)
+            if not data:
+                response = self.build_response(400, "Bad Request")  # user did fucky-wucky
+            elif len(data) > 8192:
+                response = self.build_response(413, "Request too long")
+            else:
+                response = self.handle_request(request, addr)
 
             if isinstance(response, str):
                 response = response.encode()
@@ -341,12 +426,7 @@ class WebServer:
             conn.close()
 
     def handle_request(self, data, addr):
-        print(f"len data: {len(data)}")
-        if not data:
-            return self.build_response(400, "Bad Request")  # user did fucky-wucky
-        if len(data) > 8192:
-            return self.build_response(413, "Request too long")
-
+        print(f"data: {data}")
         request_line = data.splitlines()[0]
 
         # Extract host from headers, never works though
@@ -364,17 +444,32 @@ class WebServer:
                 400, self.no_host_req_response.encode()
             )
 
+        for line in data.splitlines():
+            if "User-Agent" in line:
+                ua = line.split(":", 1)[1].strip()
+                allowed = self.parser.ua_blocker(ua)
+                if not allowed:
+                    return self.build_response(
+                        403, "This UA has been blocked by the owner of this site."
+                    )
+                break
+        else:
+            return self.build_response(
+                400, "You cannot connect without a User-Agent."
+            )
+
         method, path, version = self.parser.parse_request_line(request_line)
 
+        if not all([method, path, version]):
+            return self.build_response(400, "Bad Request")
+
         # Figure out a better way to reload config
         if path == "/?pywebsrv_reload_conf=1":
             print("Got reload command! Reloading configuration...")
-            self.file_handler.base_dir = self.file_handler.read_config("directory")
+            self.file_handler = FileHandler()
+            self.parser = RequestParser()
             return self.build_response(302, "")
 
-        if not all([method, path, version]):
-            return self.build_response(400, "Bad Request")
-
         if not self.parser.is_method_allowed(
             method
         ):
@@ -397,6 +492,9 @@ class WebServer:
         # A really crude implementation of binary files. Later in 2.0 I'll actually
         # make this useful.
         mimetype = mimetype[0]
+        if mimetype is None:
+            # We have to assume it's binary.
+            return self.build_binary_response(200, file_content, "application/octet-stream")
         if "text/" not in mimetype:
             return self.build_binary_response(200, file_content, mimetype)
 
@@ -410,17 +508,18 @@ class WebServer:
             403: "Forbidden",
             404: "Not Found",
             405: "Method Not Allowed",
-            500: "Internal Server Error",
+            500: "Internal Server Error"
         }
         status_message = messages.get(status_code)
         headers = (
             f"HTTP/1.1 {status_code} {status_message}\r\n"
-            f"Server: PyWebServer/1.2.1\r\n"
+            f"Server: PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}\r\n"
             f"Content-Type: {content_type}\r\n"
             f"Content-Length: {len(binary_data)}\r\n"
             f"Connection: close\r\n\r\n"
             # Connection close is done because it is way easier to implement.
             # It's not like this program will see production use anyway.
+            # Tbh when i'll implement HTTP2
         )
         return headers.encode() + binary_data
 
@@ -440,7 +539,7 @@ class WebServer:
             405: "Method Not Allowed",
             413: "Payload Too Large",
             500: "Internal Server Error",
-            635: "Go Away",
+            635: "Go Away"
         }
         status_message = messages.get(status_code)
 
@@ -451,7 +550,7 @@ class WebServer:
         # Don't encode yet, if 302 status code we have to include location.
         headers = (
             f"HTTP/1.1 {status_code} {status_message}\r\n"
-            f"Server: PyWebServer/1.2.1\r\n"
+            f"Server: PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}\r\n"
             f"Content-Length: {len(body)}\r\n"
             f"Connection: close\r\n\r\n"
         ).encode()
@@ -475,7 +574,7 @@ class WebServer:
             headers = (
                 f"HTTP/1.1 {status_code} {status_message}\r\n"
                 f"Location: {host}\r\n"
-                f"Server: PyWebServer/1.2.1\r\n"
+                f"Server: PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}\r\n"
                 f"Content-Length: {len(body)}\r\n"
                 f"Connection: close\r\n\r\n"
             ).encode()
@@ -491,13 +590,25 @@ class WebServer:
 
 
 def main():
+    print(
+        "WARNING!!\n"
+        f"This is Amethyst alpha build {AMETHYST_BUILD_NUMBER}\n"
+        "Since this is an alpha version of Amethyst, most features aren't working!\n"
+        "These builds are also very verbose and will spit out a lot on the terminal. "
+        "As you can imagine, this is for debugging purposes.\n"
+        "THERE IS ABSOLUTELY NO SUPPORT FOR THESE VERSIONS!\n"
+        "DO NOT USE THEM IN PRODUCTION SETTINGS!\n"
+        f"Please report any bugs on {AMETHYST_REPO}\n"
+    )
+    input("Press <Enter> to continue. ")
     file_handler = FileHandler()
-    file_handler.check_first_run()
     file_handler.base_dir = file_handler.read_config("directory")
-    http_port = file_handler.read_config("port") or 8080
-    https_port = file_handler.read_config("port-https") or 8443
-    http_enabled = file_handler.read_config("http") or True
-    https_enabled = file_handler.read_config("https") or False
+    http_port = file_handler.read_new_config("port") or 8080
+    https_port = file_handler.read_new_config("port-https") or 8443
+    http_enabled = bool(file_handler.read_new_config("http")) or True
+    print(http_enabled)
+    https_enabled = bool(file_handler.read_new_config("https")) or False
+    print(https_enabled)
     server = WebServer(http_port=http_port, https_port=https_port)
     server.start(http_enabled, https_enabled)