Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3276224943 | ||
|
|
bd78ab9225 | ||
|
|
095f516a55 |
@@ -0,0 +1,21 @@
|
|||||||
|
# Changelog from 0.2.0 to 0.3.0
|
||||||
|
|
||||||
|
## Major changes
|
||||||
|
|
||||||
|
* Improved multithreading engine to be actually multithreaded
|
||||||
|
|
||||||
|
* Gave the Python extension a beter name
|
||||||
|
|
||||||
|
* Getting ready for feature-freeze.
|
||||||
|
|
||||||
|
## Minor changes
|
||||||
|
|
||||||
|
* Fixed bugs pertaining to proxy
|
||||||
|
|
||||||
|
* Attempted fix at hanging socket by introducing a default 25 second timeout.
|
||||||
|
|
||||||
|
## Configuration changes
|
||||||
|
|
||||||
|
* Added a new `threading` key to define if threading should be enabled.
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# PES
|
||||||
|
This is a quick reference document on how to implement PES.
|
||||||
|
## Example script:
|
||||||
|
The default script is the following:
|
||||||
|
```python
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
|
||||||
|
if not os.getcwd() in sys.path:
|
||||||
|
sys.path.append(os.getcwd())
|
||||||
|
import amethyst
|
||||||
|
|
||||||
|
|
||||||
|
class PES:
|
||||||
|
"""
|
||||||
|
class
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
# DO NOT USE THIS CLASS FOR PROGRAM, ONLY ON_REQUEST PLEASE!!
|
||||||
|
# Below go definitions to get things working.
|
||||||
|
self.build_response = amethyst.WebServer.build_binary_response
|
||||||
|
self.fh = amethyst.FileHandler("..")
|
||||||
|
self.rq = amethyst.RequestParser()
|
||||||
|
# NOTE: THREAD_SAFETY is a required setting, as it defines
|
||||||
|
# if it can run within the Amethyst thread pool or needs to
|
||||||
|
# run independently
|
||||||
|
# False = run independent. True = run in Amethyst thread pool.
|
||||||
|
self.THREAD_SAFETY: bool = False
|
||||||
|
|
||||||
|
def on_request(self, req):
|
||||||
|
return self.build_response(200, "This is a test", "text/html")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
# Code to run if it is not thread-safe.
|
||||||
|
PES.on_request(PES, "request")
|
||||||
|
```
|
||||||
@@ -1,12 +1,18 @@
|
|||||||
# Amethyst Web Server
|
# Amethyst Web Server
|
||||||
|
|
||||||
## A word of warning!
|
## A word of warning!
|
||||||
|
|
||||||
Currently Amethyst is in very early alpha stage, a lot of things will be broken, names won't be correct,
|
Currently Amethyst is in very early alpha stage, a lot of things will be broken, names won't be correct,
|
||||||
promised features missing, but I'm very much working on it live!
|
promised features missing, but I'm very much working on it live!
|
||||||
Every save I do increments the build number by 1, I won't publish all of them, but most of them will be published.
|
Every save I do increments the build number by 1, I won't publish all of them, but some of them will be published.
|
||||||
Once a milestone is hit (e.g. a new feature fully implemented), I'll publish a release!
|
Once a milestone is hit (e.g. a new feature fully implemented), I'll publish a release!
|
||||||
|
|
||||||
|
## Approaching 1.0.0!
|
||||||
|
|
||||||
|
Amethyst is finally approaching 1.0.0! Very very soon I will feature-freeze the project and begin just fixing bugs and cleaning up code! This may take a bit because the codebase is very cluttered, and because all features are there in a basic state, it would be better to fix and clean up what I have, so I have a workable codebase for implementing new features, and because new features aren't going to be added anyway, I might as well fully release the project!
|
||||||
|
|
||||||
## Currently working features:
|
## Currently working features:
|
||||||
|
|
||||||
* New configuration is ~95% done, most features work.
|
* New configuration is ~95% done, most features work.
|
||||||
* Fixed **A LOT** of unreported bugs from the old code.
|
* Fixed **A LOT** of unreported bugs from the old code.
|
||||||
* More resilliency against errors.
|
* More resilliency against errors.
|
||||||
@@ -14,20 +20,96 @@ Once a milestone is hit (e.g. a new feature fully implemented), I'll publish a r
|
|||||||
* Proxy almost working!
|
* Proxy almost working!
|
||||||
|
|
||||||
## Project status:
|
## Project status:
|
||||||
|
|
||||||
Amethyst will stay in beta for a while, I want all features to work, but I will make pre-release versions that are mostly stable.
|
Amethyst will stay in beta for a while, I want all features to work, but I will make pre-release versions that are mostly stable.
|
||||||
They can be found as the `amethyst-prerel-0.a.b` releases. I won't guarantee 100% stability, but waay more than just some random build.
|
They can be found as the `amethyst-prerel-0.a.b` releases. I won't guarantee 100% stability, but waay more than just some random build.
|
||||||
|
|
||||||
## Install instructions:
|
## Install instructions:
|
||||||
|
|
||||||
Install Python, execute `amethyst.py` and change the provided config.
|
Install Python, execute `amethyst.py` and change the provided config.
|
||||||
|
|
||||||
## Minimum requirements:
|
## Minimum requirements:
|
||||||
Python 3.8+
|
|
||||||
|
Python 3.10+
|
||||||
And whatever PC that happens to run that.
|
And whatever PC that happens to run that.
|
||||||
I recommend Python 3.10 or above though, with a PC running:
|
I recommend Python 3.12 or above though, with a PC running:
|
||||||
|
|
||||||
* Windows 8.1+
|
* Windows 8.1+
|
||||||
* macOS 10.15+
|
* macOS 10.15+
|
||||||
* Linux 4.19+
|
* Linux 4.19+
|
||||||
* FreeBSD 13.2R+
|
* FreeBSD 13.2R+
|
||||||
* Some other somewhat recent OS.
|
* Some other somewhat recent OS.
|
||||||
|
|
||||||
## Currently W.I.P. Check back later!
|
## The webserver itself:
|
||||||
|
|
||||||
|
The Amethyst webserver is meant to be easy to use and configure. Its configuration takes inspiration from nginx and Caddyfile.
|
||||||
|
The language the configuration is made in is AmethystConf.
|
||||||
|
The default config is as follows:
|
||||||
|
|
||||||
|
```amethystconf
|
||||||
|
host * {
|
||||||
|
directory:./html
|
||||||
|
pesmode:0
|
||||||
|
index:index.html
|
||||||
|
}
|
||||||
|
|
||||||
|
globals {
|
||||||
|
http:1
|
||||||
|
https:1
|
||||||
|
port:8080
|
||||||
|
https-port:8443
|
||||||
|
key:./key.pem
|
||||||
|
cert:./cert.pem
|
||||||
|
max-length:8192
|
||||||
|
threading:1
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
It uses a key-value syntax, and uses a `:` as its seperator. A few key directives:
|
||||||
|
`host`, followed by a hostname signifies a host that will be available. Similar to nginx's `server_name` directive.
|
||||||
|
`globals` signifies all values that are of global importance, like the key and certificate file.
|
||||||
|
`directory` signifies the directory to look in for files on that specific host.
|
||||||
|
`index`, while not in the default config, signifies what path should be returned if the client only asks for `/` (or any subpaths without files).
|
||||||
|
`pesmode` signifies if the PES mode must be enabled, allowing the server to run custom Python code to manipulate the request or file further.
|
||||||
|
`block-ua` signifies if a specific (or loosely matched) User-Agent must be blocked from accessing the site.
|
||||||
|
`proxy` signifies if a the server needs to get the response from a different (remote) server but still needs to be available at this host.
|
||||||
|
`max-length` signifies the maximum length a request may have.
|
||||||
|
`threading` signifies if the threading engine should be enabled. This will lend more performance, but should be disabled if you use scripts that are not thread-safe.
|
||||||
|
AmethystConf has only 4 datatypes: `String`, `Boolean`, `Function` and `None`. A quick rundown:
|
||||||
|
`String` is the everything datatype. Everything is assumed to be a `String` unless it falls under the other categories.
|
||||||
|
`Boolean` is the datatype used to enable/disable features. A `Boolean` can have one of two possible values: `1` or `0`.
|
||||||
|
`Function` is the datatype used in `match()`, it signifies that the parser has to do some work on this string before it can use it.
|
||||||
|
`None` is the datatype assigned to any key without a value.
|
||||||
|
|
||||||
|
## PES (Python Extension Script)
|
||||||
|
|
||||||
|
The PES (Python Extension Script) is one of Amethysts main selling points. It's a new type of a dynamic page. A PES file is pretty much a
|
||||||
|
Python script with some conventions. Currently it is in very alpha form. It will be heavily improved upon to make sure even people with no
|
||||||
|
Python knowledge can work with it. Here's how it works:
|
||||||
|
If PES mode is enabled, the request is sent to the `pes.py` script, more specifically, the `on_request(req)` function of the `PES()` class.
|
||||||
|
From there, the decoded request is given to you to play with. All of Amethysts request and file processing tools are available, and soon
|
||||||
|
a function will be added to hand the request back to Amethyst, if it is deemed not suitable for PES mode.
|
||||||
|
Once you're done manipulating the request, all you have to do is call `return self.build_response(http_status_code, resp_body, mimetype)` and Amethyst will handle the rest.
|
||||||
|
A few examples of what can be achieved with PES mode without writing any other language than Python, HTML and CSS:
|
||||||
|
|
||||||
|
* Showing a random image from the `/pics` folder upon requesting `/randompic` from the server
|
||||||
|
* Dynamically updating the time on a website
|
||||||
|
* Create a complex calculator
|
||||||
|
* Upload files to the server
|
||||||
|
* Lock down webpages with a login prompt.
|
||||||
|
* And much, much more.
|
||||||
|
|
||||||
|
While it might not be able to create truly dynamic pages (since PES runs server-side, not client-side), it is dynamically static, basically, it's a dynamic page until it's rendered
|
||||||
|
in browser, where it's static, as PES cannot change anything there.
|
||||||
|
**WARNING!**
|
||||||
|
PES is an advanced feature! You can absolutely compromise the security of your webserver by having a misconfigured PES file. While Amethyst still has a few protection measures built-in
|
||||||
|
that activate before any request reaches the PES, but some are bypassed unless manually invoked in the PES. Because of that, here's a general user advisory:
|
||||||
|
|
||||||
|
* Use `self.fh.read_file(file_path, host=None)` instead of `open(file_path)` because of file inclusion or directory traversal concerns.
|
||||||
|
* Use `self.fh.write_file(file_path, host=None)` instead of `open(file_path)` because of file inclusion or directory traversal concerns.
|
||||||
|
* **NEVER** allow the PES to run shell code!
|
||||||
|
* **NEVER** allow the PES to run **ANYTHING** uploaded via the internet!
|
||||||
|
* Try running as much of the code locally, getting data from the internet can not only take long, it can also pose a security risk.
|
||||||
|
|
||||||
|
The PES will **NOT** warn you if you have security issues, it's a very hands-off approach. The PES will happily run `sudo rm -rf / --no-preserve-root` if given the command and
|
||||||
|
setup for shell execution and not tell you until everything is gone. Prevent those scenarios by limiting what PES does as much as possible!
|
||||||
|
|||||||
+5
-3
@@ -2,8 +2,8 @@
|
|||||||
|
|
||||||
host * {
|
host * {
|
||||||
directory:./html
|
directory:./html
|
||||||
apimode:0
|
pesmode:0
|
||||||
block-ua:match("Discordbot")
|
index:index.html
|
||||||
}
|
}
|
||||||
|
|
||||||
globals {
|
globals {
|
||||||
@@ -12,5 +12,7 @@ globals {
|
|||||||
port:8080
|
port:8080
|
||||||
https-port:8443
|
https-port:8443
|
||||||
key:./key.pem
|
key:./key.pem
|
||||||
cert./cert.pem
|
cert:./cert.pem
|
||||||
|
max-length:8192
|
||||||
|
threading:1
|
||||||
}
|
}
|
||||||
|
|||||||
+207
-87
@@ -61,7 +61,7 @@ except ImportError:
|
|||||||
)
|
)
|
||||||
# pass
|
# pass
|
||||||
|
|
||||||
AMETHYST_BUILD_NUMBER = "b0.2.0-0072"
|
AMETHYST_BUILD_NUMBER = "0.3.0-0114-mt-tryout1"
|
||||||
AMETHYST_REPO = "https://git.novacow.ch/Nova/PyWebServer/"
|
AMETHYST_REPO = "https://git.novacow.ch/Nova/PyWebServer/"
|
||||||
|
|
||||||
|
|
||||||
@@ -111,11 +111,15 @@ class ConfigParser:
|
|||||||
|
|
||||||
def query_config(self, key, host=None):
|
def query_config(self, key, host=None):
|
||||||
if host:
|
if host:
|
||||||
return self.data["hosts"].get(host, {}).get(key)
|
value = self.data["hosts"].get(host, {}).get(key)
|
||||||
if key == "hosts":
|
elif key == "hosts":
|
||||||
print(f"\n\n\nHosts!\nHosts: {self.data['hosts']}\n\n\n")
|
print(f"\n\n\nHosts!\nHosts: {self.data['hosts']}\n\n\n")
|
||||||
return list(self.data["hosts"].keys())
|
value = list(self.data["hosts"].keys())
|
||||||
return self.data["globals"].get(key)
|
else:
|
||||||
|
value = self.data["globals"].get(key)
|
||||||
|
if value == "0" or value == "1":
|
||||||
|
value = int(value)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
class FileHandler:
|
class FileHandler:
|
||||||
@@ -139,7 +143,7 @@ class FileHandler:
|
|||||||
def read_file(self, file_path, directory=None):
|
def read_file(self, file_path, directory=None):
|
||||||
if "../" in file_path or "%" in file_path:
|
if "../" in file_path or "%" in file_path:
|
||||||
return 403, None
|
return 403, None
|
||||||
if file_path == "api.py":
|
if file_path == "pes.py":
|
||||||
return 404, None
|
return 404, None
|
||||||
|
|
||||||
if directory is not None:
|
if directory is not None:
|
||||||
@@ -161,7 +165,7 @@ class FileHandler:
|
|||||||
if "../" in file_path or "%" in file_path:
|
if "../" in file_path or "%" in file_path:
|
||||||
return 403
|
return 403
|
||||||
full_path = os.path.join(self.base_dir, file_path.lstrip("/"))
|
full_path = os.path.join(self.base_dir, file_path.lstrip("/"))
|
||||||
with open(full_path, "a") as f:
|
with open(full_path, "wb") as f:
|
||||||
f.write(data)
|
f.write(data)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
@@ -186,13 +190,23 @@ class RequestParser:
|
|||||||
self.hosts = self.file_handler.read_config("hosts")
|
self.hosts = self.file_handler.read_config("hosts")
|
||||||
print(f"Hosts: {self.hosts}")
|
print(f"Hosts: {self.hosts}")
|
||||||
|
|
||||||
def parse_request_line(self, line, host):
|
def extract_header(self, header: str, request: bytes | str):
|
||||||
|
if isinstance(request, bytes):
|
||||||
|
request = request.decode("iso-8859-1", "ignore")
|
||||||
|
lines = request.splitlines()
|
||||||
|
for line in lines:
|
||||||
|
if line.startswith(header):
|
||||||
|
value = line.split(":")[1][1:]
|
||||||
|
return value
|
||||||
|
return None
|
||||||
|
|
||||||
|
def parse_request_line(self, line, host, no_mod=False):
|
||||||
"""Parses the HTTP request line."""
|
"""Parses the HTTP request line."""
|
||||||
try:
|
try:
|
||||||
method, path, version = line.split(" ")
|
method, path, version = line.split(" ")
|
||||||
except ValueError:
|
except ValueError:
|
||||||
return None, None, None
|
return None, None, None
|
||||||
if path.endswith("/") or ("." not in path):
|
if (path.endswith("/") or ("." not in path)) and (not no_mod):
|
||||||
if not path.endswith("/"):
|
if not path.endswith("/"):
|
||||||
path += "/"
|
path += "/"
|
||||||
index = self.file_handler.read_config("index", host) or "index.html"
|
index = self.file_handler.read_config("index", host) or "index.html"
|
||||||
@@ -262,20 +276,42 @@ class RequestParser:
|
|||||||
class ProxyServer:
|
class ProxyServer:
|
||||||
def __init__(self, fh):
|
def __init__(self, fh):
|
||||||
self.file_handler: FileHandler = fh
|
self.file_handler: FileHandler = fh
|
||||||
|
self.rq: RequestParser = RequestParser()
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def recv_all(sock):
|
||||||
|
chunks = []
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
data = sock.recv(4096)
|
||||||
|
if not data:
|
||||||
|
break
|
||||||
|
chunks.append(data)
|
||||||
|
except socket.timeout:
|
||||||
|
break
|
||||||
|
return b"".join(chunks)
|
||||||
|
|
||||||
def try_connection(
|
def try_connection(
|
||||||
self, host: str, port: int, data: bytes, chost: str, force_tls: bool = None
|
self, host: str, port: int, data: bytes, chost: str, force_tls: bool = None
|
||||||
):
|
):
|
||||||
if port in [443, 8443, 9443]:
|
|
||||||
do_tls = True
|
|
||||||
else:
|
|
||||||
if force_tls is True:
|
|
||||||
do_tls = True
|
|
||||||
else:
|
|
||||||
do_tls = False
|
|
||||||
print(f"\n\n\nchost: {chost}\n\n\n")
|
print(f"\n\n\nchost: {chost}\n\n\n")
|
||||||
nhost = self.file_handler.read_config("proxy", chost)
|
nhost = self.file_handler.read_config("proxy", chost)
|
||||||
print(f"\n\n\nnhost: {nhost}\n\n\n")
|
print(f"\n\n\nnhost: {nhost}\n\n\n")
|
||||||
|
# nhost will include http or https.
|
||||||
|
if nhost.startswith("https"):
|
||||||
|
nhost = nhost[6:-1]
|
||||||
|
do_tls = True
|
||||||
|
elif nhost.startswith("http"):
|
||||||
|
nhost = nhost[5:-1]
|
||||||
|
do_tls = False
|
||||||
|
else:
|
||||||
|
raise SyntaxError(
|
||||||
|
"Syntax error in config! Key: `proxy` Reason: `Expected http([...]) or https([...]), not "
|
||||||
|
f"{nhost[:6]}[...]{nhost[-1:]}!`"
|
||||||
|
)
|
||||||
|
if force_tls is True:
|
||||||
|
do_tls = True
|
||||||
|
print(f"\n\n\nnhost: {nhost}\n\n\n")
|
||||||
if ":" in nhost:
|
if ":" in nhost:
|
||||||
nport = int(nhost.split(":")[1])
|
nport = int(nhost.split(":")[1])
|
||||||
nhost = nhost.split(":")[0]
|
nhost = nhost.split(":")[0]
|
||||||
@@ -284,47 +320,52 @@ class ProxyServer:
|
|||||||
print(f"{nhost}, {nport}, {data}")
|
print(f"{nhost}, {nport}, {data}")
|
||||||
data = self.reset_host(nhost, nport, data)
|
data = self.reset_host(nhost, nport, data)
|
||||||
try:
|
try:
|
||||||
return self.tcp_send(host, port, data, do_tls)
|
print("Waiting on TCP start.")
|
||||||
except Exception:
|
return self.tcp_send(nhost, nport, data, do_tls)
|
||||||
if do_tls is False:
|
except Exception as e:
|
||||||
print("Retrying with TLS...")
|
raise Exception(f"Server replied unexpected. Reply from Python subsystem: {e}")
|
||||||
return self.try_connection(host, port, data, chost, True)
|
|
||||||
else:
|
|
||||||
raise
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def reset_host(host: str, port: int, data: bytes):
|
def reset_host(host: str, port: int, data: bytes):
|
||||||
data = data.decode()
|
header_end = data.find(b"\r\n\r\n")
|
||||||
data = data.splitlines()
|
|
||||||
for line in data:
|
if header_end == -1:
|
||||||
print(line)
|
return data
|
||||||
if line.startswith("Host:"):
|
|
||||||
|
header_bytes = data[:header_end]
|
||||||
|
body = data[header_end + 4:]
|
||||||
|
|
||||||
|
headers = header_bytes.decode("iso-8859-1")
|
||||||
|
|
||||||
|
lines = headers.split("\r\n")
|
||||||
|
new_lines = []
|
||||||
|
|
||||||
|
for line in lines:
|
||||||
|
lower = line.lower()
|
||||||
|
|
||||||
|
if lower.startswith("host:"):
|
||||||
if port not in [80, 443]:
|
if port not in [80, 443]:
|
||||||
new_line = f"Host: {host}:{port}"
|
line = f"Host: {host}:{port}"
|
||||||
else:
|
else:
|
||||||
new_line = f"Host: {host}"
|
line = f"Host: {host}"
|
||||||
idx = data.index(line)
|
|
||||||
data[idx] = new_line
|
elif lower.startswith("connection:"):
|
||||||
print(f"\n\n\n{idx}\n\n\n")
|
line = "Connection: close"
|
||||||
if line.startswith("Connection:"):
|
|
||||||
idx = data.index(line)
|
new_lines.append(line)
|
||||||
new_line = "Connection: close"
|
|
||||||
data[idx] = new_line
|
rebuilt_headers = "\r\n".join(new_lines).encode("iso-8859-1")
|
||||||
data = "\r\n".join(data)
|
|
||||||
data = f"{data}\r\n\r\n"
|
return rebuilt_headers + b"\r\n\r\n" + body
|
||||||
print(data)
|
|
||||||
return data.encode()
|
|
||||||
# return data
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def create_tls_context():
|
def create_tls_context():
|
||||||
# Create a context that by default verifies with system CAs
|
|
||||||
ctx = ssl.create_default_context()
|
ctx = ssl.create_default_context()
|
||||||
ctx.check_hostname = False
|
ctx.check_hostname = False
|
||||||
ctx.verify_mode = ssl.CERT_NONE
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
return ctx
|
return ctx
|
||||||
|
|
||||||
def tcp_send(self, host, port, data: bytes, do_tls: bool):
|
def tcp_send(self, host, port, data: bytes, do_tls: booll):
|
||||||
try:
|
try:
|
||||||
with socket.create_connection((host, port), timeout=10) as raw_sock:
|
with socket.create_connection((host, port), timeout=10) as raw_sock:
|
||||||
raw_sock.settimeout(10)
|
raw_sock.settimeout(10)
|
||||||
@@ -336,10 +377,36 @@ class ProxyServer:
|
|||||||
) as ssock:
|
) as ssock:
|
||||||
ssock.sendall(data)
|
ssock.sendall(data)
|
||||||
print("data reached")
|
print("data reached")
|
||||||
return ssock.recv(512000)
|
resp = self.recv_all(ssock)
|
||||||
|
if self.rq.extract_header("Transfer-Encoding", resp) == "chunked":
|
||||||
|
ssock.sendall(b"TRANSER-ENCODING IS NOT SUPPORTED")
|
||||||
|
ssock.close()
|
||||||
|
resp = (
|
||||||
|
"HTTP/1.1 502 Gateway Error\r\nConnection: close\r\nContent-Length: 270\r\n"
|
||||||
|
f"Server: Amethyst/build-{AMETHYST_BUILD_NUMBER}\r\n\r\n"
|
||||||
|
"Gateway Error.\nThe upstream server tried to use a a transfer mode not "
|
||||||
|
"yet supported\nExact error:\nE_DATA_STREAMING_NOT_SUPPORTED\n"
|
||||||
|
"The unsupported mode:\nTransfer-Encoding: chunked.\n"
|
||||||
|
"There is no fix. The problem lies with the proxy, and is not a fault of the upstream server."
|
||||||
|
)
|
||||||
|
return resp
|
||||||
else:
|
else:
|
||||||
|
print(f"\n\n\nraw data: {data}\n\n\n")
|
||||||
raw_sock.sendall(data)
|
raw_sock.sendall(data)
|
||||||
return raw_sock.recv(512000)
|
print("Waiting for response...")
|
||||||
|
resp = self.recv_all(raw_sock)
|
||||||
|
if self.rq.extract_header("Transfer-Encoding", resp) is not None:
|
||||||
|
raw_sock.sendall(b"TRANSER-ENCODING IS NOT SUPPORTED")
|
||||||
|
raw_sock.close()
|
||||||
|
resp = (
|
||||||
|
"HTTP/1.1 502 Gateway Error\r\nConnection: close\r\nContent-Length: 270\r\n"
|
||||||
|
f"Server: Amethyst/build-{AMETHYST_BUILD_NUMBER}\r\n\r\n"
|
||||||
|
"Gateway Error.\nThe upstream server tried to use a a transfer mode not "
|
||||||
|
"yet supported\nExact error:\nE_DATA_STREAMING_NOT_SUPPORTED\n"
|
||||||
|
"The unsupported mode:\nTransfer-Encoding: chunked.\n"
|
||||||
|
"There is no fix. The problem lies with the proxy, and is not a fault of the upstream server."
|
||||||
|
)
|
||||||
|
return resp
|
||||||
except Exception:
|
except Exception:
|
||||||
raise
|
raise
|
||||||
|
|
||||||
@@ -354,6 +421,7 @@ class WebServer:
|
|||||||
self.parser = RequestParser()
|
self.parser = RequestParser()
|
||||||
self.cert_file = self.file_handler.read_config("cert") or cert_file
|
self.cert_file = self.file_handler.read_config("cert") or cert_file
|
||||||
self.key_file = self.file_handler.read_config("key") or key_file
|
self.key_file = self.file_handler.read_config("key") or key_file
|
||||||
|
self.max_length = int(self.file_handler.read_config("max-length")) or 8192
|
||||||
self.skip_ssl = False
|
self.skip_ssl = False
|
||||||
|
|
||||||
# me when no certificate and key file
|
# me when no certificate and key file
|
||||||
@@ -382,9 +450,11 @@ class WebServer:
|
|||||||
|
|
||||||
self.http_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
|
self.http_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
|
||||||
self.http_socket.bind(("::", self.http_port))
|
self.http_socket.bind(("::", self.http_port))
|
||||||
|
self.http_socket.settimeout(25)
|
||||||
|
|
||||||
self.https_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
|
self.https_socket_raw = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
|
||||||
self.https_socket.bind(("::", self.https_port))
|
self.https_socket_raw.bind(("::", self.https_port))
|
||||||
|
self.https_socket_raw.settimeout(25)
|
||||||
|
|
||||||
self.proxy_handler = ProxyServer(self.file_handler)
|
self.proxy_handler = ProxyServer(self.file_handler)
|
||||||
|
|
||||||
@@ -395,7 +465,7 @@ class WebServer:
|
|||||||
certfile=self.cert_file, keyfile=self.key_file
|
certfile=self.cert_file, keyfile=self.key_file
|
||||||
)
|
)
|
||||||
self.https_socket = self.ssl_context.wrap_socket(
|
self.https_socket = self.ssl_context.wrap_socket(
|
||||||
self.https_socket, server_side=True
|
self.https_socket_raw, server_side=True
|
||||||
)
|
)
|
||||||
|
|
||||||
self.http_404_html = (
|
self.http_404_html = (
|
||||||
@@ -417,24 +487,19 @@ class WebServer:
|
|||||||
self.running = True
|
self.running = True
|
||||||
|
|
||||||
def start(self, http, https):
|
def start(self, http, https):
|
||||||
signal.signal(signal.SIGINT, self.shutdown)
|
|
||||||
signal.signal(signal.SIGTERM, self.shutdown)
|
|
||||||
|
|
||||||
http_thread = threading.Thread(target=self.start_http, daemon=True)
|
|
||||||
https_thread = threading.Thread(target=self.start_https, daemon=True)
|
|
||||||
|
|
||||||
if https is True:
|
if https is True:
|
||||||
if self.skip_ssl is True:
|
if self.skip_ssl is True:
|
||||||
print("WARN: You have enabled HTTPS without SSL!!")
|
print("WARN: You have enabled HTTPS without SSL!!")
|
||||||
yn = input("Is this intended behaviour? [y/N] ")
|
yn = input("Is this intended behaviour? [y/N] ")
|
||||||
if yn.lower() == "n":
|
if yn.lower() == "n":
|
||||||
exit(1)
|
exit(1)
|
||||||
https_thread.start()
|
self.start_https()
|
||||||
|
else:
|
||||||
|
self.https_socket.close()
|
||||||
if http is True:
|
if http is True:
|
||||||
http_thread.start()
|
self.start_http()
|
||||||
|
else:
|
||||||
http_thread.join()
|
self.http_socket.close()
|
||||||
https_thread.join()
|
|
||||||
|
|
||||||
def start_http(self):
|
def start_http(self):
|
||||||
self.http_socket.listen(5)
|
self.http_socket.listen(5)
|
||||||
@@ -442,11 +507,19 @@ class WebServer:
|
|||||||
while self.running:
|
while self.running:
|
||||||
try:
|
try:
|
||||||
conn, addr = self.http_socket.accept()
|
conn, addr = self.http_socket.accept()
|
||||||
|
if self.file_handler.read_config("threading") is True:
|
||||||
|
threading.Thread(
|
||||||
|
target=self.handle_connection,
|
||||||
|
args=(conn, addr),
|
||||||
|
daemon=True
|
||||||
|
).start()
|
||||||
|
else:
|
||||||
self.handle_connection(conn, addr)
|
self.handle_connection(conn, addr)
|
||||||
except Exception as e:
|
|
||||||
print(f"HTTP error: {e}")
|
|
||||||
except OSError:
|
except OSError:
|
||||||
break
|
break
|
||||||
|
except Exception as e:
|
||||||
|
if not "timeout" in f"{e}":
|
||||||
|
print(f"HTTP error: {e}")
|
||||||
|
|
||||||
def start_https(self):
|
def start_https(self):
|
||||||
self.https_socket.listen(5)
|
self.https_socket.listen(5)
|
||||||
@@ -454,23 +527,58 @@ class WebServer:
|
|||||||
while self.running:
|
while self.running:
|
||||||
try:
|
try:
|
||||||
conn, addr = self.https_socket.accept()
|
conn, addr = self.https_socket.accept()
|
||||||
|
if self.file_handler.read_config("threading") is True:
|
||||||
|
threading.Thread(
|
||||||
|
target=self.handle_connection,
|
||||||
|
args=(conn, addr),
|
||||||
|
daemon=True
|
||||||
|
).start()
|
||||||
|
else:
|
||||||
self.handle_connection(conn, addr)
|
self.handle_connection(conn, addr)
|
||||||
except Exception as e:
|
|
||||||
print(
|
|
||||||
f"HTTPS error: {e}"
|
|
||||||
) # be ready for ssl errors if you use a self-sign!!
|
|
||||||
except OSError:
|
except OSError:
|
||||||
break
|
break
|
||||||
|
except Exception as e:
|
||||||
|
if not "timeout" in f"{e}":
|
||||||
|
print(f"HTTPS error: {e}")
|
||||||
|
|
||||||
def handle_connection(self, conn, addr):
|
def handle_connection(self, conn, addr):
|
||||||
try:
|
try:
|
||||||
data = conn.recv(32768)
|
data = b""
|
||||||
request = data.decode(errors="ignore")
|
# Read headers
|
||||||
|
while b"\r\n\r\n" not in data:
|
||||||
|
chunk = conn.recv(4096)
|
||||||
|
if not chunk:
|
||||||
|
break
|
||||||
|
data += chunk
|
||||||
|
|
||||||
|
headers, _, rest = data.partition(b"\r\n\r\n")
|
||||||
|
|
||||||
|
# Parse Content-Length
|
||||||
|
content_length = 0
|
||||||
|
for line in headers.split(b"\r\n"):
|
||||||
|
if line.lower().startswith(b"content-length:"):
|
||||||
|
content_length = int(line.split(b":")[1].strip())
|
||||||
|
|
||||||
|
# print(f"Content-Length to server: {content_length}")
|
||||||
|
|
||||||
|
# Read body
|
||||||
|
body = rest
|
||||||
|
print(f"Rest length: {len(rest)}")
|
||||||
|
while len(body) < content_length:
|
||||||
|
chunk = conn.recv(4096)
|
||||||
|
# print(f"\n\nrecv returned {len(chunk)}\n\n")
|
||||||
|
if not chunk:
|
||||||
|
print("\n\nsocket closed\n\n")
|
||||||
|
break
|
||||||
|
body += chunk
|
||||||
|
|
||||||
|
data += body
|
||||||
|
request = data.decode("iso-8859-1", errors="ignore")
|
||||||
if not data:
|
if not data:
|
||||||
response = self.build_response(
|
response = self.build_response(
|
||||||
400, "Bad Request"
|
400, "Bad Request"
|
||||||
) # user did fucky-wucky
|
) # user did fucky-wucky
|
||||||
elif len(data) > 8192:
|
elif len(data) > self.max_length:
|
||||||
response = self.build_response(413, "Request too long")
|
response = self.build_response(413, "Request too long")
|
||||||
else:
|
else:
|
||||||
response = self.handle_request(request, addr)
|
response = self.handle_request(request, addr)
|
||||||
@@ -493,7 +601,7 @@ class WebServer:
|
|||||||
conn.close()
|
conn.close()
|
||||||
|
|
||||||
def handle_request(self, data, addr):
|
def handle_request(self, data, addr):
|
||||||
print(f"data: {data}")
|
# print(f"data: {data}")
|
||||||
request_line = data.splitlines()[0]
|
request_line = data.splitlines()[0]
|
||||||
|
|
||||||
# Extract host from headers, never works though
|
# Extract host from headers, never works though
|
||||||
@@ -539,14 +647,14 @@ class WebServer:
|
|||||||
value = self.file_handler.read_config("proxy", host)
|
value = self.file_handler.read_config("proxy", host)
|
||||||
if ":" in value:
|
if ":" in value:
|
||||||
host = value.split(":")[0]
|
host = value.split(":")[0]
|
||||||
port = int(value.split(":")[1])
|
port = int(value.split(":")[1][:-1])
|
||||||
else:
|
else:
|
||||||
host = value
|
host = value
|
||||||
port = 443
|
port = 443
|
||||||
return self.proxy_handler.try_connection(
|
return self.proxy_handler.try_connection(
|
||||||
host,
|
host,
|
||||||
port,
|
port,
|
||||||
data.encode(),
|
data.encode("iso-8859-1"),
|
||||||
orig_host,
|
orig_host,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -565,13 +673,25 @@ class WebServer:
|
|||||||
or self.file_handler.base_dir
|
or self.file_handler.base_dir
|
||||||
)
|
)
|
||||||
|
|
||||||
if self.file_handler.read_config("apimode", host) is True:
|
if bool(self.file_handler.read_config("pesmode", host)) is True:
|
||||||
if not os.path.join(os.getcwd(), directory) in sys.path:
|
if not os.path.join(os.getcwd(), directory) in sys.path:
|
||||||
sys.path.append(f"{os.path.join(os.getcwd(), directory)}")
|
sys.path.append(f"{os.path.join(os.getcwd(), directory)}")
|
||||||
import api
|
import pes
|
||||||
|
try:
|
||||||
|
pesclass = pes.PES()
|
||||||
|
threadcompat = pesclass.THREAD_SAFETY
|
||||||
|
# if not threadcompat:
|
||||||
|
return pesclass.on_request(data)
|
||||||
|
except Exception as e:
|
||||||
|
return self.build_response(
|
||||||
|
500,
|
||||||
|
"Amethyst is currently unable to serve your request. Below is debug info.\r\n"
|
||||||
|
f"Error: {e}; Version: amethyst-b{AMETHYST_BUILD_NUMBER}\r\n"
|
||||||
|
"You cannot do anything at this time, the server owner has made a misconfiguration in their Python Extension Script",
|
||||||
|
)
|
||||||
|
|
||||||
apiclass = api.API()
|
if bool(self.file_handler.read_config("621mode", host)) is True:
|
||||||
return apiclass.on_request(data)
|
return self.build_response(621, "")
|
||||||
|
|
||||||
file_content, mimetype = self.file_handler.read_file(path, directory)
|
file_content, mimetype = self.file_handler.read_file(path, directory)
|
||||||
|
|
||||||
@@ -671,14 +791,14 @@ class WebServer:
|
|||||||
|
|
||||||
if status_code == 621:
|
if status_code == 621:
|
||||||
headers = (
|
headers = (
|
||||||
f"HTTP/1.1 {status_code} {status_message}\r\n"
|
"HTTP/1.1 302 UwU Nya!\r\n"
|
||||||
"Server: Amethyst/build-0621\r\n"
|
"Server: Amethyst/build-0621\r\n"
|
||||||
"Content-Length: 30\r\n"
|
"Content-Length: 0\r\n"
|
||||||
f"Connection: close\r\n\r\n"
|
"Connection: close\r\n"
|
||||||
)
|
"Note: congrats, you found a funny. i guess.\r\n"
|
||||||
body = "https://e621.net/posts/6155664"
|
"Host: https://e621.net/posts/\r\n\r\n"
|
||||||
|
).encode("iso-8859-1")
|
||||||
print(f"{headers + body}")
|
body = "".encode("iso-8859-1")
|
||||||
return headers + body
|
return headers + body
|
||||||
|
|
||||||
def shutdown(self, signum, frame):
|
def shutdown(self, signum, frame):
|
||||||
@@ -703,11 +823,11 @@ def main():
|
|||||||
input("Press <Enter> to continue. ")
|
input("Press <Enter> to continue. ")
|
||||||
file_handler = FileHandler()
|
file_handler = FileHandler()
|
||||||
file_handler.base_dir = file_handler.read_config("directory")
|
file_handler.base_dir = file_handler.read_config("directory")
|
||||||
http_port = file_handler.read_config("port") or 8080
|
http_port = file_handler.read_config("port")
|
||||||
https_port = file_handler.read_config("https-port") or 8443
|
https_port = file_handler.read_config("https-port")
|
||||||
http_enabled = bool(file_handler.read_config("http")) or True
|
http_enabled = bool(file_handler.read_config("http"))
|
||||||
print(http_enabled)
|
print(http_enabled)
|
||||||
https_enabled = bool(file_handler.read_config("https")) or False
|
https_enabled = bool(file_handler.read_config("https"))
|
||||||
print(https_enabled)
|
print(https_enabled)
|
||||||
server = WebServer(http_port=http_port, https_port=https_port)
|
server = WebServer(http_port=http_port, https_port=https_port)
|
||||||
server.start(http_enabled, https_enabled)
|
server.start(http_enabled, https_enabled)
|
||||||
|
|||||||
+34
-16
@@ -6,28 +6,50 @@ import datetime
|
|||||||
|
|
||||||
|
|
||||||
class AutoCertGen:
|
class AutoCertGen:
|
||||||
def __init__(self):
|
def __init__(self, name="website", org="organization", locale="place", province="province", country="ZZ", dns="localhost"):
|
||||||
pass
|
self.name = name
|
||||||
|
self.org = org
|
||||||
|
self.locale = locale
|
||||||
|
self.province = province
|
||||||
|
self.country = country
|
||||||
|
self.dns = dns
|
||||||
|
|
||||||
def gen_cert(self):
|
def generate_self_signed_cert(self, different_issuer=False):
|
||||||
# Generate private key
|
|
||||||
private_key = rsa.generate_private_key(
|
private_key = rsa.generate_private_key(
|
||||||
public_exponent=65537,
|
public_exponent=65537,
|
||||||
key_size=2048,
|
key_size=2048,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Define subject and issuer (self-signed)
|
if different_issuer is True:
|
||||||
|
subject = x509.Name(
|
||||||
|
[
|
||||||
|
x509.NameAttribute(NameOID.COUNTRY_NAME, self.country),
|
||||||
|
x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, self.province),
|
||||||
|
x509.NameAttribute(NameOID.LOCALITY_NAME, self.locale),
|
||||||
|
x509.NameAttribute(NameOID.ORGANIZATION_NAME, self.org),
|
||||||
|
x509.NameAttribute(NameOID.COMMON_NAME, self.name),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
issuer = x509.Name(
|
||||||
|
[
|
||||||
|
x509.NameAttribute(NameOID.COUNTRY_NAME, "RU"),
|
||||||
|
x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, "Красноярск Область"),
|
||||||
|
x509.NameAttribute(NameOID.LOCALITY_NAME, "Красноярск"),
|
||||||
|
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "Amethyst Group"),
|
||||||
|
x509.NameAttribute(NameOID.COMMON_NAME, "Amethyst Untrusted Signing Certificate"),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
else:
|
||||||
subject = issuer = x509.Name(
|
subject = issuer = x509.Name(
|
||||||
[
|
[
|
||||||
x509.NameAttribute(NameOID.COUNTRY_NAME, "ZZ"),
|
x509.NameAttribute(NameOID.COUNTRY_NAME, self.country),
|
||||||
x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, "Some province"),
|
x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, self.province),
|
||||||
x509.NameAttribute(NameOID.LOCALITY_NAME, "Some place"),
|
x509.NameAttribute(NameOID.LOCALITY_NAME, self.locale),
|
||||||
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "Some org"),
|
x509.NameAttribute(NameOID.ORGANIZATION_NAME, self.org),
|
||||||
x509.NameAttribute(NameOID.COMMON_NAME, "localhost"),
|
x509.NameAttribute(NameOID.COMMON_NAME, self.name),
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|
||||||
# Create certificate
|
|
||||||
certificate = (
|
certificate = (
|
||||||
x509.CertificateBuilder()
|
x509.CertificateBuilder()
|
||||||
.subject_name(subject)
|
.subject_name(subject)
|
||||||
@@ -39,12 +61,11 @@ class AutoCertGen:
|
|||||||
datetime.datetime.utcnow() + datetime.timedelta(days=365)
|
datetime.datetime.utcnow() + datetime.timedelta(days=365)
|
||||||
) # 1 year validity
|
) # 1 year validity
|
||||||
.add_extension(
|
.add_extension(
|
||||||
x509.SubjectAlternativeName([x509.DNSName("localhost")]), critical=False
|
x509.SubjectAlternativeName([x509.DNSName(self.dns)]), critical=False
|
||||||
)
|
)
|
||||||
.sign(private_key, hashes.SHA256())
|
.sign(private_key, hashes.SHA256())
|
||||||
)
|
)
|
||||||
|
|
||||||
# Save private key
|
|
||||||
with open("key.pem", "wb") as f:
|
with open("key.pem", "wb") as f:
|
||||||
f.write(
|
f.write(
|
||||||
private_key.private_bytes(
|
private_key.private_bytes(
|
||||||
@@ -54,8 +75,5 @@ class AutoCertGen:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
# Save certificate
|
|
||||||
with open("cert.pem", "wb") as f:
|
with open("cert.pem", "wb") as f:
|
||||||
f.write(certificate.public_bytes(serialization.Encoding.PEM))
|
f.write(certificate.public_bytes(serialization.Encoding.PEM))
|
||||||
|
|
||||||
print("Self-signed certificate and private key generated for HTTPS server!")
|
|
||||||
|
|||||||
+1
-1
@@ -8,7 +8,7 @@
|
|||||||
<h1>Hello from Amethyst!</h1>
|
<h1>Hello from Amethyst!</h1>
|
||||||
<h2>This page confirms Amethyst can read files from your PC or server and serve them to your browser!</h2>
|
<h2>This page confirms Amethyst can read files from your PC or server and serve them to your browser!</h2>
|
||||||
<p>This is a test page, if you aren't the server owner, they might not have finished setting up their site, be patient. If this doesn't go away after a while, tell them they've made an oopsie</p>
|
<p>This is a test page, if you aren't the server owner, they might not have finished setting up their site, be patient. If this doesn't go away after a while, tell them they've made an oopsie</p>
|
||||||
<p>This server runs Amethyst Pre-Rel 0.2.0-0072</p>
|
<p>This server runs Amethyst build 0.3.0-0114-mt-tryout1</p>
|
||||||
</center>
|
</center>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|||||||
@@ -9,10 +9,10 @@ import os
|
|||||||
|
|
||||||
if not os.getcwd() in sys.path:
|
if not os.getcwd() in sys.path:
|
||||||
sys.path.append(os.getcwd())
|
sys.path.append(os.getcwd())
|
||||||
import pywebsrv
|
import amethyst
|
||||||
|
|
||||||
|
|
||||||
class API:
|
class PES:
|
||||||
"""
|
"""
|
||||||
class
|
class
|
||||||
"""
|
"""
|
||||||
@@ -20,7 +20,10 @@ class API:
|
|||||||
def __init__(self):
|
def __init__(self):
|
||||||
# DO NOT USE THIS CLASS FOR PROGRAM, ONLY ON_REQUEST PLEASE!!
|
# DO NOT USE THIS CLASS FOR PROGRAM, ONLY ON_REQUEST PLEASE!!
|
||||||
# Below go definitions to get things working.
|
# Below go definitions to get things working.
|
||||||
self.build_response = pywebsrv.WebServer.build_binary_response
|
self.build_response = amethyst.WebServer.build_binary_response
|
||||||
|
self.fh = amethyst.FileHandler("..")
|
||||||
|
self.rq = amethyst.RequestParser()
|
||||||
|
self.THREAD_SAFETY: bool = True
|
||||||
|
|
||||||
def on_request(self, req):
|
def on_request(self, req):
|
||||||
return self.build_response(200, "This is a test", "text/html")
|
return self.build_response(200, b"Heyhey! This is the default PES script!", "text/html")
|
||||||
Reference in New Issue
Block a user