Update README.md

README.md

@@ -1,9 +1,97 @@
-# Amethyst Web Server
+# PyWebServer
 
-## A word of warning!
+## Current state
-Currently Amethyst is in very early alpha stage, a lot of things will be broken, names won't be correct,
+Currently I'm in the middle of bringing major improvements to PyWebServer, like a significantly better config
-promised features missing, but I'm very much working on it live!  
+automatic HTTPS certificate generation using Let's Encrypt, HTTP `PUT`, `POST` and `DELETE` support,
-Every save I do increments the build number by 1, I won't publish all of them, but most of them will be published.  
+HTTP2 support and auto-indexing.  
-Once a milestone is hit (e.g. a new feature fully implemented), I'll publish a release!
+This all will be part of major release 2, version 2.0. These improvements will partially make their way into 1.x
+versions, from 1.4 onwards, mostly as testing ground and compatibility with 2.0 reasons. Most will probably be dead code
+until 2.0 properly comes out. Until then, parts of the code may be pretty cluttered. I'll probably enable you to test some feautres
+like the new config, but code quality is currently not a main priority.
 
-## Currently W.I.P. Check back later!
+## GitHub
+The host of this project is on my own [Gitea instance](https://git.novacow.ch/Nova/PyWebServer/).  
+Because of that I'll mostly reply to issues and PRs there, you can submit issues and PRs on GitHub, but it might take longer before I read it.
+
+## Installing
+### The little tiny easier route
+Installing and running PyWebServer is very simple.  
+First off, download the latest release from the 'Releases' tab, choose the Zip variant if unsure.  
+When it's done downloading, unpack the files in a directory of choice, for the purpose of this README, 
+I've chosen `./pywebserver/` (for Windows: `.\pywebserver\`).  
+From there, open up your favorite text editor and open the file `pywebsrv.conf` in the directory you unpacked PyWebServer.  
+In there, you should see this somewhere:
+```
+# Here you choose what directory PyWebServer looks in for files.
+directory:<Enter directory here>
+```
+After the colon, enter your directory where you have your website stored.  
+After that, make sure you have installed Python. Here's how you can install Python:  
+Linux:
+```bash
+sudo apt install python3  # Debian / Ubuntu
+sudo dnf install python3  # Fedora / Nobara
+sudo pacman -S python3  # Arch and derivatives.
+```
+macOS:
+```bash
+brew install python3
+```
+Windows:
+```powershell
+# You can change the `3.12` with whatever version you want/need.
+winget install -e --id Python.Python.3.12 --scope machine
+```
+Then, in the terminal window you have open, go to the directory you unpacked PyWebServer and type this:
+```
+python3 ./pywebsrv.py
+# For Windows users, if the above command doesn't work, try this:
+py ./pywebsrv.py
+```
+And there you go! You've now set up PyWebServer!
+
+### The little tiny harder route
+Installing and running PyWebServer is very simple.  
+Assuming you're running Linux:
+```bash
+git clone https://git.novacow.ch/Nova/PyWebServer.git
+cd ./PyWebServer/
+```
+Windows users, make sure you have installed Git, from there:
+```powershell
+git clone https://git.novacow.ch/Nova/PyWebServer.git
+Set-Location .\PyWebServer\
+```
+Then, open `pywebsrv.conf` in your favorite text editor and change the `directory` key to the full path where your files are stored.  
+After that, put your files in and run this:
+Linux:
+```bash
+python3 /path/to/pywebsrv.py
+```
+Windows:
+```powershell
+# If you have installed Python via the Microsoft Store:
+python3 \path\to\pywebsrv.py
+# Via the python.org website:
+py \path\to\pywebsrv.py
+```
+
+## SSL Support
+PyWebServer supports SSL/TLS for authentication via HTTPS. In the config file, you should enable the HTTPS port. After that you need to create the certificate.  
+Currently PyWebServer looks for the `cert.pem` and the `key.pem` files in the root directory of the installation.  
+
+## HTTP support
+Currently PyWebServer only supports HTTP/1.1, this is very unlikely to change, as most of the modern web today still uses HTTP/1.1.  
+For methods PyWebServer only supports `GET`, this is being reworked though, check issue [#3](https://git.novacow.ch/Nova/PyWebServer/issues/3) for progress.
+
+## Files support
+Unlike other small web servers, PyWebServer has full support for binary files being sent and received (once that logic is put in) over HTTP(S).
+
+## Support
+PyWebServer will follow a standard support scheme.
+### 1.x
+For every 1.x version there will be support until 2 newer versions come out.
+So that means that 1.0 will still be supported when 1.1 comes out, but no longer be supported when 1.2 comes out.
+### 2.x
+I am planning on releasing a 2.x version with will have a lot more advanced features, like nginx's server block emulation amongst other things.
+When 2.0 will come out, 1.x will be developed further, but 2.0 will be the main focus.

html/index.html

@@ -2,20 +2,12 @@
 <html>
 <head>
     <title>Test page</title>
-    <link rel="stylesheet" href="/style.css">
 </head>
 <body>
-    <center>
+    <h1>Hey there!</h1>
-    <h1>Web portal</h1>
+    <h2>You're seeing this page because you haven't set up PyWebServer yet!</h2>
-    <h1><a href="/d.mp4">d.mp4</a></h1>
+    <h2>This page confirms that PyWebServer can read and serve files from your PC.</h2>
-    <h1><a href="/webjammies/index.ini">index.ini</a></h1>
+    <h2>To make this go away, please edit the file `pywebsrv.conf` and edit the `directory` key to your directory of choice!</h2>
-    <h1><a href="/webjammies/rabado.mp3">rabado.mp3</a></h1>
+    <p>Here you can simulate a 404 error: <a href="/uuh">Click me for a 404 error!</a></p>
-    <h1><a href="/webjammies/invaders.mp3">invaders.mp3</a></h1>
-    <h1><a href="/webjammies/momentum.mp3">momentum.mp3</a></h1>
-    <h1><a href="/webjammies/bacterial-love.mp3">bacterial-love.mp3</a></h1>
-    <h1><a href="/webjammies/old-money-bitch.mp3">old-money-bitch.mp3</a></h1>
-    <h1><a href="/webjammies/ravestate909.mp3">ravestate909.mp3</a></h1>
-    <h1><a href="/webjammies/leash.mp3">leash.mp3</a></h1>
-    </center>
 </body>
 </html>

new_conf.conf

@@ -1,36 +0,0 @@
-# WARNING: This is an alpha spec of NSCL 2.0!!
-
-host 192.168.1.196 {
-    location /secrets {
-        root:/var/www/html/hidden
-        whitelist-ua:"Mozilla/5.0 (X11; Linux x86_64) Gecko/20100101 Firefox/149.0 Furfox/1.0"
-        whitelist-ip:123.45.67.89
-    }
-    directory:/home/nova/Downloads/test/html
-    allowed-methods:GET
-    block-ip:match-ip("192.168",2)
-    block-ua:match("Discordbot",0),match("Google",0)
-}
-
-host cdn.example.com {
-    directory:/home/nova/Downloads/test/cdn
-    allowed-methods:GET,PUT
-    block-ip:10.1.100.2
-    block-ua:match("Discordbot",0)
-}
-
-host modem.example.com {
-    proxy:192.168.2.254
-    key-file:/home/nova/Downloads/test/proxykey.pem
-    cert-file:/home/nova/Downloads/test/proxycert.pem
-}
-
-globals {
-    http:1
-    https:1
-    port:8080
-    https-port:8443
-    allow-localhost:1
-    global-key:/home/nova/Downloads/test/key.pem
-    global-cert:/home/nova/Downloads/test/cert.pem
-}

pywebsrv.conf

@@ -4,11 +4,11 @@
 port:8080
 port-https:8443
 # Here you choose what directory PyWebServer looks in for files.
-directory:/home/nova/PyWebServer/html
+directory:<Enter directory here>
 # Host defenition, what hosts you can connect via.
 # You can use FQDNs, IP-addresses and localhost,
 # Support for multiple hosts is coming.
-host:localhost,10.185.213.118
+host:localhost
 # Enables HTTP support. (Only enables/disables the HTTP port.)
 http:1
 # Enables HTTPS support. (Only enables/disables the HTTPS port.)
@@ -21,8 +21,8 @@ allow-localhost:1
 disable-autocertgen:0
 # If you wish to block IP-addresses, this function is coming though.
 # block-ip:0.0.0.0,1.1.1.1,2.2.2.2
-# If you wish to block User-Agents.
+# If you wish to block User-Agents, this function is coming though.
-block-ua:match(Discordbot),match(google)
+# block-ua:(NULL)
 
 # TEST: experimental non-defined keys go here:
 # keyfile key

pywebsrv.py

@@ -56,17 +56,12 @@ except ImportError:
     # )
     pass
 
-AMETHYST_BUILD_NUMBER = "0018"
-AMETHYST_REPO = "https://git.novacow.ch/Nova/PyWebServer/"
-
 
 class FileHandler:
     CONFIG_FILE = "pywebsrv.conf"
-    new_conf = "new_conf.conf"
 
     def __init__(self, base_dir=None):
         self.config_path = os.path.join(os.getcwd(), self.CONFIG_FILE)
-        self.new_conf = os.path.join(os.getcwd(), self.new_conf)
         self.base_dir = self.read_config("directory")
         self.cached_conf = None
         if not os.path.exists(self.config_path):
@@ -77,13 +72,10 @@ class FileHandler:
             )
             exit(1)
 
-    def read_file(self, file_path, directory=None):
+    def read_file(self, file_path):
-        if "../" in file_path or "%" in file_path:
+        if "../" in file_path:
             return 403, None
 
-        if directory is not None:
-            full_path = os.path.join(directory, file_path.lstrip("/"))
-        else:
         full_path = os.path.join(self.base_dir, file_path.lstrip("/"))
         if not os.path.isfile(full_path):
             return 404, None
@@ -96,8 +88,8 @@ class FileHandler:
             print(f"Error reading file {full_path}: {e}")
             return 500, None
 
-    def write_file(self, file_path, data, directory=None):
+    def write_file(self, file_path, data):
-        if "../" in file_path or "%" in file_path:
+        if "../" in file_path:
             return 403
         full_path = os.path.join(self.base_dir, file_path.lstrip("/"))
         with open(full_path, "a") as f:
@@ -174,7 +166,7 @@ class FileHandler:
         Reads the configuration file and returns a dict
         """
         if self.cached_conf is None:
-            with open(self.new_conf, "r", encoding="utf-8") as fh:
+            with open(self.config_path, "r", encoding="utf-8") as fh:
                 text = fh.read()
 
             blocks = re.findall(
@@ -182,7 +174,6 @@ class FileHandler:
             )
             parsed = {}
             host_list = []
-            print(f"Blocks: {blocks}")
             for tag, hostname, body in blocks:
                 section = hostname if hostname else "globals"
                 if hostname:
@@ -190,13 +181,14 @@ class FileHandler:
                 kv = {}
                 for line in body.splitlines():
                     line = line.strip()
-                    if not line or ":" not in line or line.startswith("#"):
+                    if not line or ":" not in line or line.starswith("#"):
                         continue
 
                     key, rest = line.split(":", 1)
                     key = key.strip()
                     rest = rest.strip()
 
+                    # Split comma-separated values (e.g. GET,PUT)
                     if "," in rest:
                         kv[key] = [item.strip() for item in rest.split(",")]
                     else:
@@ -219,8 +211,6 @@ class FileHandler:
         Generate some self-signed certificates using AutoCertGen
         TODO: doesn't work, need to fix. probably add `./` to $PATH
         """
-        if not os.getcwd() in sys.path:
-            sys.path.append(f"{os.getcwd()}")
         autocert = AutoCertGen()
         autocert.gen_cert()
 
@@ -228,8 +218,7 @@ class FileHandler:
 class RequestParser:
     def __init__(self):
         self.file_handler = FileHandler()
-        self.hosts = self.file_handler.read_new_config("host")
+        self.hosts = self.file_handler.read_config("host")
-        print(f"Hosts: {self.hosts}")
 
     def parse_request_line(self, line):
         """Parses the HTTP request line."""
@@ -241,9 +230,8 @@ class RequestParser:
             path += "index.html"
         return method, path, version
 
-    def ua_blocker(self, ua, host=None):
+    def ua_blocker(self, ua):
         """Parses and matches UA to block"""
-        del host
         match, literal = self.file_handler.read_config("block-ua")
         if ua in literal:
             return False
@@ -272,14 +260,13 @@ class RequestParser:
         Mfw im in an ugly code writing contest and my opponent is nova while writing a side project
         """
         host = f"{host}"
-        print(f"hosts: {self.hosts}, host: {host}, split: {host.rsplit(":", 1)[0]}")
         if ":" in host:
-            host = host.rsplit(":", 1)[0]
+            host = host.split(":", 1)[0]
         host = host.lstrip()
         host = host.rstrip()
         if (
-            host == "localhost" or host == "127.0.0.1" or host == "[::1]"
+            host == "localhost" or host == "127.0.0.1"
-        ) and self.file_handler.read_new_config("allow-localhost"):
+        ) and self.file_handler.read_config("allow-localhost"):
             return True
         if host not in self.hosts:
             return False
@@ -297,8 +284,8 @@ class WebServer:
     def __init__(
         self, http_port=8080, https_port=8443, cert_file="cert.pem", key_file="key.pem"
     ):
-        self.http_port = int(http_port)
+        self.http_port = http_port
-        self.https_port = int(https_port)
+        self.https_port = https_port
         self.cert_file = cert_file
         self.key_file = key_file
         self.file_handler = FileHandler()
@@ -329,22 +316,14 @@ class WebServer:
             "This host cannot be reached without sending a `Host` header."
         )
 
-        # TODO: enable experimental ipv6 support in config
-
         # ipv6 when????/??//?????//?
-        # self.http_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        self.http_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        # self.http_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+        self.http_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-        # self.http_socket.bind(("0.0.0.0", self.http_port))
+        self.http_socket.bind(("0.0.0.0", self.http_port))
-        #
-        # self.https_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        # self.https_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-        # self.https_socket.bind(("0.0.0.0", self.https_port))
 
-        self.http_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
+        self.https_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
-        self.http_socket.bind(("::", self.http_port))
+        self.https_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
-
+        self.https_socket.bind(("0.0.0.0", self.https_port))
-        self.https_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
-        self.https_socket.bind(("::", self.https_port))
 
         if self.skip_ssl is False:
             # https gets the ssl treatment!! yaaaay :3
@@ -358,17 +337,17 @@ class WebServer:
 
         self.http_404_html = (
             "<html><head><title>HTTP 404 - PyWebServer</title></head>"
-            f"<body><center><h1>HTTP 404 - Not Found!</h1><p>Running PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}</p>"
+            "<body><center><h1>HTTP 404 - Not Found!</h1><p>Running PyWebServer/1.2.1</p>"
             "</center></body></html>"
         )
         self.http_403_html = (
             "<html><head><title>HTTP 403 - PyWebServer</title></head>"
-            f"<body><center><h1>HTTP 403 - Forbidden</h1><p>Running PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}</p>"
+            "<body><center><h1>HTTP 403 - Forbidden</h1><p>Running PyWebServer/1.2.1</p>"
             "</center></body></html>"
         )
         self.http_405_html = (
             "<html><head><title>HTTP 405 - PyWebServer</title></head>"
-            f"<body><center><h1>HTTP 405 - Method not allowed</h1><p>Running PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}</p>"
+            "<body><center><h1>HTTP 405 - Method not allowed</h1><p>Running PyWebServer/1.2.1</p>"
             "</center></body></html>"
         )
 
@@ -380,23 +359,17 @@ class WebServer:
 
         http_thread = threading.Thread(target=self.start_http, daemon=True)
         https_thread = threading.Thread(target=self.start_https, daemon=True)
-        # ipv6http_thread = threading.Thread(target=self.start_http_ipv6, daemon=True)
-        # ipv6https_thread = threading.Thread(target=self.start_https_ipv6, daemon=True)
 
         if https is True:
             if self.skip_ssl is True:
                 print("WARN: You have enabled HTTPS without SSL!!")
                 yn = input("Is this intended behaviour? [y/N] ")
             https_thread.start()
-            # ipv6https_thread.start()
         if http is True:
-            # ipv6http_thread.start()
             http_thread.start()
 
         http_thread.join()
         https_thread.join()
-        # ipv6http_thread.join()
-        # ipv6https_thread.join()
 
     def start_http(self):
         self.http_socket.listen(5)
@@ -410,32 +383,6 @@ class WebServer:
             except OSError:
                 break
 
-    def start_http_ipv6(self):
-        self.ipv6http_socket.listen(5)
-        print(f"IPv6 HTTP server listening on port {self.http_port}...")
-        while self.running:
-            try:
-                conn, addr = self.ipv6http_socket.accept()
-                self.handle_connection(conn, addr)
-            except Exception as e:
-                print(f"HTTP error: {e}")
-            except OSError:
-                break
-
-    def start_https_ipv6(self):
-        self.ipv6https_socket.listen(5)
-        print(f"IPv6 HTTPS server listening on port {self.https_port}...")
-        while self.running:
-            try:
-                conn, addr = self.ipv6https_socket.accept()
-                self.handle_connection(conn, addr)
-            except Exception as e:
-                print(
-                    f"HTTPS error: {e}"
-                )  # be ready for ssl errors if you use a self-sign!!
-            except OSError:
-                break
-
     def start_https(self):
         self.https_socket.listen(5)
         print(f"HTTPS server listening on port {self.https_port}...")
@@ -520,9 +467,7 @@ class WebServer:
         ):
             return self.build_response(405, self.http_405_html)
 
-        directory = self.file_handler.read_new_config("directory", host)
+        file_content, mimetype = self.file_handler.read_file(path)
-
-        file_content, mimetype = self.file_handler.read_file(path, directory)
 
         if file_content == 403:
             print("WARN: Directory traversal attack prevented.")  # look ma, security!!
@@ -560,7 +505,7 @@ class WebServer:
         status_message = messages.get(status_code)
         headers = (
             f"HTTP/1.1 {status_code} {status_message}\r\n"
-            f"Server: PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}\r\n"
+            f"Server: PyWebServer/1.4\r\n"
             f"Content-Type: {content_type}\r\n"
             f"Content-Length: {len(binary_data)}\r\n"
             f"Connection: close\r\n\r\n"
@@ -597,7 +542,7 @@ class WebServer:
         # Don't encode yet, if 302 status code we have to include location.
         headers = (
             f"HTTP/1.1 {status_code} {status_message}\r\n"
-            f"Server: PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}\r\n"
+            f"Server: PyWebServer/1.4\r\n"
             f"Content-Length: {len(body)}\r\n"
             f"Connection: close\r\n\r\n"
         ).encode()
@@ -621,7 +566,7 @@ class WebServer:
             headers = (
                 f"HTTP/1.1 {status_code} {status_message}\r\n"
                 f"Location: {host}\r\n"
-                f"Server: PyWebServer/amethyst-build-{AMETHYST_BUILD_NUMBER}\r\n"
+                f"Server: PyWebServer/1.2.1\r\n"
                 f"Content-Length: {len(body)}\r\n"
                 f"Connection: close\r\n\r\n"
             ).encode()
@@ -637,25 +582,13 @@ class WebServer:
 
 
 def main():
-    print(
-        "WARNING!!\n"
-        f"This is Amethyst alpha build {AMETHYST_BUILD_NUMBER}\n"
-        "Since this is an alpha version of Amethyst, most features aren't working!\n"
-        "These builds are also very verbose and will spit out a lot on the terminal. "
-        "As you can imagine, this is for debugging purposes.\n"
-        "THERE IS ABSOLUTELY NO SUPPORT FOR THESE VERSIONS!\n"
-        "DO NOT USE THEM IN PRODUCTION SETTINGS!\n"
-        f"Please report any bugs on {AMETHYST_REPO}\n"
-    )
-    input("Press <Enter> to continue. ")
     file_handler = FileHandler()
+    file_handler.check_first_run()
     file_handler.base_dir = file_handler.read_config("directory")
-    http_port = file_handler.read_new_config("port") or 8080
+    http_port = file_handler.read_config("port") or 8080
-    https_port = file_handler.read_new_config("port-https") or 8443
+    https_port = file_handler.read_config("port-https") or 8443
-    http_enabled = bool(file_handler.read_new_config("http")) or True
+    http_enabled = file_handler.read_config("http") or True
-    print(http_enabled)
+    https_enabled = file_handler.read_config("https") or False
-    https_enabled = bool(file_handler.read_new_config("https")) or False
-    print(https_enabled)
     server = WebServer(http_port=http_port, https_port=https_port)
     server.start(http_enabled, https_enabled)