amethyst 0018

Partial new config functionality.

README.md

@@ -1,97 +1,9 @@
-# PyWebServer
+# Amethyst Web Server
 
-## Current state
+## A word of warning!
-Currently I'm in the middle of bringing major improvements to PyWebServer, like a significantly better config
+Currently Amethyst is in very early alpha stage, a lot of things will be broken, names won't be correct,
-automatic HTTPS certificate generation using Let's Encrypt, HTTP `PUT`, `POST` and `DELETE` support,
+promised features missing, but I'm very much working on it live!  
-HTTP2 support and auto-indexing.  
+Every save I do increments the build number by 1, I won't publish all of them, but most of them will be published.  
-This all will be part of major release 2, version 2.0. These improvements will partially make their way into 1.x
+Once a milestone is hit (e.g. a new feature fully implemented), I'll publish a release!
-versions, from 1.4 onwards, mostly as testing ground and compatibility with 2.0 reasons. Most will probably be dead code
-until 2.0 properly comes out. Until then, parts of the code may be pretty cluttered. I'll probably enable you to test some feautres
-like the new config, but code quality is currently not a main priority.
 
-## GitHub
+## Currently W.I.P. Check back later!
-The host of this project is on my own [Gitea instance](https://git.novacow.ch/Nova/PyWebServer/).  
-Because of that I'll mostly reply to issues and PRs there, you can submit issues and PRs on GitHub, but it might take longer before I read it.
-
-## Installing
-### The little tiny easier route
-Installing and running PyWebServer is very simple.  
-First off, download the latest release from the 'Releases' tab, choose the Zip variant if unsure.  
-When it's done downloading, unpack the files in a directory of choice, for the purpose of this README, 
-I've chosen `./pywebserver/` (for Windows: `.\pywebserver\`).  
-From there, open up your favorite text editor and open the file `pywebsrv.conf` in the directory you unpacked PyWebServer.  
-In there, you should see this somewhere:
-```
-# Here you choose what directory PyWebServer looks in for files.
-directory:<Enter directory here>
-```
-After the colon, enter your directory where you have your website stored.  
-After that, make sure you have installed Python. Here's how you can install Python:  
-Linux:
-```bash
-sudo apt install python3  # Debian / Ubuntu
-sudo dnf install python3  # Fedora / Nobara
-sudo pacman -S python3  # Arch and derivatives.
-```
-macOS:
-```bash
-brew install python3
-```
-Windows:
-```powershell
-# You can change the `3.12` with whatever version you want/need.
-winget install -e --id Python.Python.3.12 --scope machine
-```
-Then, in the terminal window you have open, go to the directory you unpacked PyWebServer and type this:
-```
-python3 ./pywebsrv.py
-# For Windows users, if the above command doesn't work, try this:
-py ./pywebsrv.py
-```
-And there you go! You've now set up PyWebServer!
-
-### The little tiny harder route
-Installing and running PyWebServer is very simple.  
-Assuming you're running Linux:
-```bash
-git clone https://git.novacow.ch/Nova/PyWebServer.git
-cd ./PyWebServer/
-```
-Windows users, make sure you have installed Git, from there:
-```powershell
-git clone https://git.novacow.ch/Nova/PyWebServer.git
-Set-Location .\PyWebServer\
-```
-Then, open `pywebsrv.conf` in your favorite text editor and change the `directory` key to the full path where your files are stored.  
-After that, put your files in and run this:
-Linux:
-```bash
-python3 /path/to/pywebsrv.py
-```
-Windows:
-```powershell
-# If you have installed Python via the Microsoft Store:
-python3 \path\to\pywebsrv.py
-# Via the python.org website:
-py \path\to\pywebsrv.py
-```
-
-## SSL Support
-PyWebServer supports SSL/TLS for authentication via HTTPS. In the config file, you should enable the HTTPS port. After that you need to create the certificate.  
-Currently PyWebServer looks for the `cert.pem` and the `key.pem` files in the root directory of the installation.  
-
-## HTTP support
-Currently PyWebServer only supports HTTP/1.1, this is very unlikely to change, as most of the modern web today still uses HTTP/1.1.  
-For methods PyWebServer only supports `GET`, this is being reworked though, check issue [#3](https://git.novacow.ch/Nova/PyWebServer/issues/3) for progress.
-
-## Files support
-Unlike other small web servers, PyWebServer has full support for binary files being sent and received (once that logic is put in) over HTTP(S).
-
-## Support
-PyWebServer will follow a standard support scheme.
-### 1.x
-For every 1.x version there will be support until 2 newer versions come out.
-So that means that 1.0 will still be supported when 1.1 comes out, but no longer be supported when 1.2 comes out.
-### 2.x
-I am planning on releasing a 2.x version with will have a lot more advanced features, like nginx's server block emulation amongst other things.
-When 2.0 will come out, 1.x will be developed further, but 2.0 will be the main focus.

html/index.html

@@ -2,12 +2,20 @@
 <html>
 <head>
     <title>Test page</title>
+    <link rel="stylesheet" href="/style.css">
 </head>
 <body>
-    <h1>Hey there!</h1>
+    <center>
-    <h2>You're seeing this page because you haven't set up PyWebServer yet!</h2>
+    <h1>Web portal</h1>
-    <h2>This page confirms that PyWebServer can read and serve files from your PC.</h2>
+    <h1><a href="/d.mp4">d.mp4</a></h1>
-    <h2>To make this go away, please edit the file `pywebsrv.conf` and edit the `directory` key to your directory of choice!</h2>
+    <h1><a href="/webjammies/index.ini">index.ini</a></h1>
-    <p>Here you can simulate a 404 error: <a href="/uuh">Click me for a 404 error!</a></p>
+    <h1><a href="/webjammies/rabado.mp3">rabado.mp3</a></h1>
+    <h1><a href="/webjammies/invaders.mp3">invaders.mp3</a></h1>
+    <h1><a href="/webjammies/momentum.mp3">momentum.mp3</a></h1>
+    <h1><a href="/webjammies/bacterial-love.mp3">bacterial-love.mp3</a></h1>
+    <h1><a href="/webjammies/old-money-bitch.mp3">old-money-bitch.mp3</a></h1>
+    <h1><a href="/webjammies/ravestate909.mp3">ravestate909.mp3</a></h1>
+    <h1><a href="/webjammies/leash.mp3">leash.mp3</a></h1>
+    </center>
 </body>
 </html>

new_conf.conf

@@ -0,0 +1,36 @@
+# WARNING: This is an alpha spec of NSCL 2.0!!
+
+host 192.168.1.196 {
+    location /secrets {
+        root:/var/www/html/hidden
+        whitelist-ua:"Mozilla/5.0 (X11; Linux x86_64) Gecko/20100101 Firefox/149.0 Furfox/1.0"
+        whitelist-ip:123.45.67.89
+    }
+    directory:/home/nova/Downloads/test/html
+    allowed-methods:GET
+    block-ip:match-ip("192.168",2)
+    block-ua:match("Discordbot",0),match("Google",0)
+}
+
+host cdn.example.com {
+    directory:/home/nova/Downloads/test/cdn
+    allowed-methods:GET,PUT
+    block-ip:10.1.100.2
+    block-ua:match("Discordbot",0)
+}
+
+host modem.example.com {
+    proxy:192.168.2.254
+    key-file:/home/nova/Downloads/test/proxykey.pem
+    cert-file:/home/nova/Downloads/test/proxycert.pem
+}
+
+globals {
+    http:1
+    https:1
+    port:8080
+    https-port:8443
+    allow-localhost:1
+    global-key:/home/nova/Downloads/test/key.pem
+    global-cert:/home/nova/Downloads/test/cert.pem
+}

pywebsrv.conf

@@ -4,11 +4,11 @@
 port:8080
 port-https:8443
 # Here you choose what directory PyWebServer looks in for files.
-directory:<Enter directory here>
+directory:/home/nova/PyWebServer/html
 # Host defenition, what hosts you can connect via.
 # You can use FQDNs, IP-addresses and localhost,
 # Support for multiple hosts is coming.
-host:localhost
+host:localhost,10.185.213.118
 # Enables HTTP support. (Only enables/disables the HTTP port.)
 http:1
 # Enables HTTPS support. (Only enables/disables the HTTPS port.)
@@ -21,8 +21,8 @@ allow-localhost:1
 disable-autocertgen:0
 # If you wish to block IP-addresses, this function is coming though.
 # block-ip:0.0.0.0,1.1.1.1,2.2.2.2
-# If you wish to block User-Agents, this function is coming though.
+# If you wish to block User-Agents.
-# block-ua:(NULL)
+block-ua:match(Discordbot),match(google)
 
 # TEST: experimental non-defined keys go here:
 # keyfile key

pywebsrv.py

@@ -56,12 +56,17 @@ except ImportError:
     # )
     pass
 
+AMETHYST_BUILD_NUMBER = "0018"
+AMETHYST_REPO = "https://git.novacow.ch/Nova/PyWebServer/"
+
 
 class FileHandler:
     CONFIG_FILE = "pywebsrv.conf"
+    new_conf = "new_conf.conf"
 
     def __init__(self, base_dir=None):
         self.config_path = os.path.join(os.getcwd(), self.CONFIG_FILE)
+        self.new_conf = os.path.join(os.getcwd(), self.new_conf)
         self.base_dir = self.read_config("directory")
         self.cached_conf = None
         if not os.path.exists(self.config_path):
@@ -72,11 +77,14 @@ class FileHandler:
             )
             exit(1)
 
-    def read_file(self, file_path):
+    def read_file(self, file_path, directory=None):
-        if "../" in file_path:
+        if "../" in file_path or "%" in file_path:
             return 403, None
 
-        full_path = os.path.join(self.base_dir, file_path.lstrip("/"))
+        if directory is not None:
+            full_path = os.path.join(directory, file_path.lstrip("/"))
+        else:
+            full_path = os.path.join(self.base_dir, file_path.lstrip("/"))
         if not os.path.isfile(full_path):
             return 404, None
 
@@ -88,8 +96,8 @@ class FileHandler:
             print(f"Error reading file {full_path}: {e}")
             return 500, None
 
-    def write_file(self, file_path, data):
+    def write_file(self, file_path, data, directory=None):
-        if "../" in file_path:
+        if "../" in file_path or "%" in file_path:
             return 403
         full_path = os.path.join(self.base_dir, file_path.lstrip("/"))
         with open(full_path, "a") as f:
@@ -166,7 +174,7 @@ class FileHandler:
         Reads the configuration file and returns a dict
         """
         if self.cached_conf is None:
-            with open(self.config_path, "r", encoding="utf-8") as fh:
+            with open(self.new_conf, "r", encoding="utf-8") as fh:
                 text = fh.read()
 
             blocks = re.findall(
@@ -174,6 +182,7 @@ class FileHandler:
             )
             parsed = {}
             host_list = []
+            print(f"Blocks: {blocks}")
             for tag, hostname, body in blocks:
                 section = hostname if hostname else "globals"
                 if hostname:
@@ -181,36 +190,37 @@ class FileHandler:
                 kv = {}
                 for line in body.splitlines():
                     line = line.strip()
-                    if not line or ":" not in line or line.starswith("#"):
+                    if not line or ":" not in line or line.startswith("#"):
                         continue
 
                     key, rest = line.split(":", 1)
                     key = key.strip()
                     rest = rest.strip()
 
-                    # Split comma-separated values (e.g. GET,PUT)
                     if "," in rest:
                         kv[key] = [item.strip() for item in rest.split(",")]
                     else:
                         kv[key] = rest
-                parsed[section] = kv
+            parsed[section] = kv
-                parsed["globals"]["hosts"] = host_list
+            parsed["globals"]["hosts"] = host_list
-                self.cached_conf = parsed
+            self.cached_conf = parsed
-            else:
+        else:
-                parsed = self.cached_conf
+            parsed = self.cached_conf
-            if option == "host":
+        if option == "host":
-                try:
+            try:
-                    return host_list
+                return host_list
-                except Exception:
+            except Exception:
-                    return parsed["globals"]["hosts"]
+                return parsed["globals"]["hosts"]
-            section = parsed.get(host or "globals", {})
+        section = parsed.get(host or "globals", {})
-            return section.get(option)
+        return section.get(option)
 
     def autocert(self):
         """
         Generate some self-signed certificates using AutoCertGen
         TODO: doesn't work, need to fix. probably add `./` to $PATH
         """
+        if not os.getcwd() in sys.path:
+            sys.path.append(f"{os.getcwd()}")
         autocert = AutoCertGen()
         autocert.gen_cert()
 
@@ -218,7 +228,8 @@ class FileHandler:
 class RequestParser:
     def __init__(self):
         self.file_handler = FileHandler()
-        self.hosts = self.file_handler.read_config("host")
+        self.hosts = self.file_handler.read_new_config("host")
+        print(f"Hosts: {self.hosts}")
 
     def parse_request_line(self, line):
         """Parses the HTTP request line."""
@@ -230,8 +241,9 @@ class RequestParser:
             path += "index.html"
         return method, path, version
 
-    def ua_blocker(self, ua):
+    def ua_blocker(self, ua, host=None):
         """Parses and matches UA to block"""
+        del host
         match, literal = self.file_handler.read_config("block-ua")
         if ua in literal:
             return False
@@ -260,13 +272,14 @@ class RequestParser:
         Mfw im in an ugly code writing contest and my opponent is nova while writing a side project
         """
         host = f"{host}"
+        print(f"hosts: {self.hosts}, host: {host}, split: {host.rsplit(":", 1)[0]}")
         if ":" in host:
-            host = host.split(":", 1)[0]
+            host = host.rsplit(":", 1)[0]
         host = host.lstrip()
         host = host.rstrip()
         if (
-            host == "localhost" or host == "127.0.0.1"
+            host == "localhost" or host == "127.0.0.1" or host == "[::1]"
-        ) and self.file_handler.read_config("allow-localhost"):
+        ) and self.file_handler.read_new_config("allow-localhost"):
             return True
         if host not in self.hosts:
             return False
@@ -284,8 +297,8 @@ class WebServer:
     def __init__(
         self, http_port=8080, https_port=8443, cert_file="cert.pem", key_file="key.pem"
     ):
-        self.http_port = http_port
+        self.http_port = int(http_port)
-        self.https_port = https_port
+        self.https_port = int(https_port)
         self.cert_file = cert_file
         self.key_file = key_file
         self.file_handler = FileHandler()
@@ -316,14 +329,22 @@ class WebServer:
             "This host cannot be reached without sending a `Host` header."
         )
 
-        # ipv6 when????/??//?????//?
+        # TODO: enable experimental ipv6 support in config
-        self.http_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.http_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-        self.http_socket.bind(("0.0.0.0", self.http_port))
 
-        self.https_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        # ipv6 when????/??//?????//?
-        self.https_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        # self.http_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.https_socket.bind(("0.0.0.0", self.https_port))
+        # self.http_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        # self.http_socket.bind(("0.0.0.0", self.http_port))
+        #
+        # self.https_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        # self.https_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        # self.https_socket.bind(("0.0.0.0", self.https_port))
+
+        self.http_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
+        self.http_socket.bind(("::", self.http_port))
+
+        self.https_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
+        self.https_socket.bind(("::", self.https_port))
 
         if self.skip_ssl is False:
             # https gets the ssl treatment!! yaaaay :3
@@ -337,17 +358,17 @@ class WebServer:
 
         self.http_404_html = (
             "<html><head><title>HTTP 404 - PyWebServer</title></head>"
-            "<body><center><h1>HTTP 404 - Not Found!</h1><p>Running PyWebServer/1.2.1</p>"
+            f"<body><center><h1>HTTP 404 - Not Found!</h1><p>Running PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}</p>"
             "</center></body></html>"
         )
         self.http_403_html = (
             "<html><head><title>HTTP 403 - PyWebServer</title></head>"
-            "<body><center><h1>HTTP 403 - Forbidden</h1><p>Running PyWebServer/1.2.1</p>"
+            f"<body><center><h1>HTTP 403 - Forbidden</h1><p>Running PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}</p>"
             "</center></body></html>"
         )
         self.http_405_html = (
             "<html><head><title>HTTP 405 - PyWebServer</title></head>"
-            "<body><center><h1>HTTP 405 - Method not allowed</h1><p>Running PyWebServer/1.2.1</p>"
+            f"<body><center><h1>HTTP 405 - Method not allowed</h1><p>Running PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}</p>"
             "</center></body></html>"
         )
 
@@ -359,17 +380,23 @@ class WebServer:
 
         http_thread = threading.Thread(target=self.start_http, daemon=True)
         https_thread = threading.Thread(target=self.start_https, daemon=True)
+        # ipv6http_thread = threading.Thread(target=self.start_http_ipv6, daemon=True)
+        # ipv6https_thread = threading.Thread(target=self.start_https_ipv6, daemon=True)
 
         if https is True:
             if self.skip_ssl is True:
                 print("WARN: You have enabled HTTPS without SSL!!")
                 yn = input("Is this intended behaviour? [y/N] ")
             https_thread.start()
+            # ipv6https_thread.start()
         if http is True:
+            # ipv6http_thread.start()
             http_thread.start()
 
         http_thread.join()
         https_thread.join()
+        # ipv6http_thread.join()
+        # ipv6https_thread.join()
 
     def start_http(self):
         self.http_socket.listen(5)
@@ -383,6 +410,32 @@ class WebServer:
             except OSError:
                 break
 
+    def start_http_ipv6(self):
+        self.ipv6http_socket.listen(5)
+        print(f"IPv6 HTTP server listening on port {self.http_port}...")
+        while self.running:
+            try:
+                conn, addr = self.ipv6http_socket.accept()
+                self.handle_connection(conn, addr)
+            except Exception as e:
+                print(f"HTTP error: {e}")
+            except OSError:
+                break
+
+    def start_https_ipv6(self):
+        self.ipv6https_socket.listen(5)
+        print(f"IPv6 HTTPS server listening on port {self.https_port}...")
+        while self.running:
+            try:
+                conn, addr = self.ipv6https_socket.accept()
+                self.handle_connection(conn, addr)
+            except Exception as e:
+                print(
+                    f"HTTPS error: {e}"
+                )  # be ready for ssl errors if you use a self-sign!!
+            except OSError:
+                break
+
     def start_https(self):
         self.https_socket.listen(5)
         print(f"HTTPS server listening on port {self.https_port}...")
@@ -467,7 +520,9 @@ class WebServer:
         ):
             return self.build_response(405, self.http_405_html)
 
-        file_content, mimetype = self.file_handler.read_file(path)
+        directory = self.file_handler.read_new_config("directory", host)
+
+        file_content, mimetype = self.file_handler.read_file(path, directory)
 
         if file_content == 403:
             print("WARN: Directory traversal attack prevented.")  # look ma, security!!
@@ -505,7 +560,7 @@ class WebServer:
         status_message = messages.get(status_code)
         headers = (
             f"HTTP/1.1 {status_code} {status_message}\r\n"
-            f"Server: PyWebServer/1.4\r\n"
+            f"Server: PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}\r\n"
             f"Content-Type: {content_type}\r\n"
             f"Content-Length: {len(binary_data)}\r\n"
             f"Connection: close\r\n\r\n"
@@ -542,7 +597,7 @@ class WebServer:
         # Don't encode yet, if 302 status code we have to include location.
         headers = (
             f"HTTP/1.1 {status_code} {status_message}\r\n"
-            f"Server: PyWebServer/1.4\r\n"
+            f"Server: PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}\r\n"
             f"Content-Length: {len(body)}\r\n"
             f"Connection: close\r\n\r\n"
         ).encode()
@@ -566,7 +621,7 @@ class WebServer:
             headers = (
                 f"HTTP/1.1 {status_code} {status_message}\r\n"
                 f"Location: {host}\r\n"
-                f"Server: PyWebServer/1.2.1\r\n"
+                f"Server: PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}\r\n"
                 f"Content-Length: {len(body)}\r\n"
                 f"Connection: close\r\n\r\n"
             ).encode()
@@ -582,13 +637,25 @@ class WebServer:
 
 
 def main():
+    print(
+        "WARNING!!\n"
+        f"This is Amethyst alpha build {AMETHYST_BUILD_NUMBER}\n"
+        "Since this is an alpha version of Amethyst, most features aren't working!\n"
+        "These builds are also very verbose and will spit out a lot on the terminal. "
+        "As you can imagine, this is for debugging purposes.\n"
+        "THERE IS ABSOLUTELY NO SUPPORT FOR THESE VERSIONS!\n"
+        "DO NOT USE THEM IN PRODUCTION SETTINGS!\n"
+        f"Please report any bugs on {AMETHYST_REPO}\n"
+    )
+    input("Press <Enter> to continue. ")
     file_handler = FileHandler()
-    file_handler.check_first_run()
     file_handler.base_dir = file_handler.read_config("directory")
-    http_port = file_handler.read_config("port") or 8080
+    http_port = file_handler.read_new_config("port") or 8080
-    https_port = file_handler.read_config("port-https") or 8443
+    https_port = file_handler.read_new_config("port-https") or 8443
-    http_enabled = file_handler.read_config("http") or True
+    http_enabled = bool(file_handler.read_new_config("http")) or True
-    https_enabled = file_handler.read_config("https") or False
+    print(http_enabled)
+    https_enabled = bool(file_handler.read_new_config("https")) or False
+    print(https_enabled)
     server = WebServer(http_port=http_port, https_port=https_port)
     server.start(http_enabled, https_enabled)