Version 0.3.0
Enhanced multithreading engine, bugs fixed and more!
This commit is contained in:
@@ -0,0 +1,21 @@
|
|||||||
|
# Changelog from 0.2.0 to 0.3.0
|
||||||
|
|
||||||
|
## Major changes
|
||||||
|
|
||||||
|
* Improved multithreading engine to be actually multithreaded
|
||||||
|
|
||||||
|
* Gave the Python extension a beter name
|
||||||
|
|
||||||
|
* Getting ready for feature-freeze.
|
||||||
|
|
||||||
|
## Minor changes
|
||||||
|
|
||||||
|
* Fixed bugs pertaining to proxy
|
||||||
|
|
||||||
|
* Attempted fix at hanging socket by introducing a default 25 second timeout.
|
||||||
|
|
||||||
|
## Configuration changes
|
||||||
|
|
||||||
|
* Added a new `threading` key to define if threading should be enabled.
|
||||||
|
|
||||||
|
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# PES
|
||||||
|
This is a quick reference document on how to implement PES.
|
||||||
|
## Example script:
|
||||||
|
The default script is the following:
|
||||||
|
```python
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
|
||||||
|
if not os.getcwd() in sys.path:
|
||||||
|
sys.path.append(os.getcwd())
|
||||||
|
import amethyst
|
||||||
|
|
||||||
|
|
||||||
|
class PES:
|
||||||
|
"""
|
||||||
|
class
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
# DO NOT USE THIS CLASS FOR PROGRAM, ONLY ON_REQUEST PLEASE!!
|
||||||
|
# Below go definitions to get things working.
|
||||||
|
self.build_response = amethyst.WebServer.build_binary_response
|
||||||
|
self.fh = amethyst.FileHandler("..")
|
||||||
|
self.rq = amethyst.RequestParser()
|
||||||
|
# NOTE: THREAD_SAFETY is a required setting, as it defines
|
||||||
|
# if it can run within the Amethyst thread pool or needs to
|
||||||
|
# run independently
|
||||||
|
# False = run independent. True = run in Amethyst thread pool.
|
||||||
|
self.THREAD_SAFETY: bool = False
|
||||||
|
|
||||||
|
def on_request(self, req):
|
||||||
|
return self.build_response(200, "This is a test", "text/html")
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
# Code to run if it is not thread-safe.
|
||||||
|
PES.on_request(PES, "request")
|
||||||
|
```
|
||||||
@@ -1,12 +1,18 @@
|
|||||||
# Amethyst Web Server
|
# Amethyst Web Server
|
||||||
|
|
||||||
## A word of warning!
|
## A word of warning!
|
||||||
|
|
||||||
Currently Amethyst is in very early alpha stage, a lot of things will be broken, names won't be correct,
|
Currently Amethyst is in very early alpha stage, a lot of things will be broken, names won't be correct,
|
||||||
promised features missing, but I'm very much working on it live!
|
promised features missing, but I'm very much working on it live!
|
||||||
Every save I do increments the build number by 1, I won't publish all of them, but some of them will be published.
|
Every save I do increments the build number by 1, I won't publish all of them, but some of them will be published.
|
||||||
Once a milestone is hit (e.g. a new feature fully implemented), I'll publish a release!
|
Once a milestone is hit (e.g. a new feature fully implemented), I'll publish a release!
|
||||||
|
|
||||||
|
## Approaching 1.0.0!
|
||||||
|
|
||||||
|
Amethyst is finally approaching 1.0.0! Very very soon I will feature-freeze the project and begin just fixing bugs and cleaning up code! This may take a bit because the codebase is very cluttered, and because all features are there in a basic state, it would be better to fix and clean up what I have, so I have a workable codebase for implementing new features, and because new features aren't going to be added anyway, I might as well fully release the project!
|
||||||
|
|
||||||
## Currently working features:
|
## Currently working features:
|
||||||
|
|
||||||
* New configuration is ~95% done, most features work.
|
* New configuration is ~95% done, most features work.
|
||||||
* Fixed **A LOT** of unreported bugs from the old code.
|
* Fixed **A LOT** of unreported bugs from the old code.
|
||||||
* More resilliency against errors.
|
* More resilliency against errors.
|
||||||
@@ -14,16 +20,20 @@ Once a milestone is hit (e.g. a new feature fully implemented), I'll publish a r
|
|||||||
* Proxy almost working!
|
* Proxy almost working!
|
||||||
|
|
||||||
## Project status:
|
## Project status:
|
||||||
|
|
||||||
Amethyst will stay in beta for a while, I want all features to work, but I will make pre-release versions that are mostly stable.
|
Amethyst will stay in beta for a while, I want all features to work, but I will make pre-release versions that are mostly stable.
|
||||||
They can be found as the `amethyst-prerel-0.a.b` releases. I won't guarantee 100% stability, but waay more than just some random build.
|
They can be found as the `amethyst-prerel-0.a.b` releases. I won't guarantee 100% stability, but waay more than just some random build.
|
||||||
|
|
||||||
## Install instructions:
|
## Install instructions:
|
||||||
|
|
||||||
Install Python, execute `amethyst.py` and change the provided config.
|
Install Python, execute `amethyst.py` and change the provided config.
|
||||||
|
|
||||||
## Minimum requirements:
|
## Minimum requirements:
|
||||||
|
|
||||||
Python 3.10+
|
Python 3.10+
|
||||||
And whatever PC that happens to run that.
|
And whatever PC that happens to run that.
|
||||||
I recommend Python 3.12 or above though, with a PC running:
|
I recommend Python 3.12 or above though, with a PC running:
|
||||||
|
|
||||||
* Windows 8.1+
|
* Windows 8.1+
|
||||||
* macOS 10.15+
|
* macOS 10.15+
|
||||||
* Linux 4.19+
|
* Linux 4.19+
|
||||||
@@ -31,14 +41,15 @@ I recommend Python 3.12 or above though, with a PC running:
|
|||||||
* Some other somewhat recent OS.
|
* Some other somewhat recent OS.
|
||||||
|
|
||||||
## The webserver itself:
|
## The webserver itself:
|
||||||
|
|
||||||
The Amethyst webserver is meant to be easy to use and configure. Its configuration takes inspiration from nginx and Caddyfile.
|
The Amethyst webserver is meant to be easy to use and configure. Its configuration takes inspiration from nginx and Caddyfile.
|
||||||
The language the configuration is made in is AmethystConf.
|
The language the configuration is made in is AmethystConf.
|
||||||
The default config is as follows:
|
The default config is as follows:
|
||||||
|
|
||||||
```amethystconf
|
```amethystconf
|
||||||
host * {
|
host * {
|
||||||
directory:./html
|
directory:./html
|
||||||
pesmode:0
|
pesmode:0
|
||||||
block-ua:match("Discordbot")
|
|
||||||
index:index.html
|
index:index.html
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -50,8 +61,10 @@ globals {
|
|||||||
key:./key.pem
|
key:./key.pem
|
||||||
cert:./cert.pem
|
cert:./cert.pem
|
||||||
max-length:8192
|
max-length:8192
|
||||||
|
threading:1
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
It uses a key-value syntax, and uses a `:` as its seperator. A few key directives:
|
It uses a key-value syntax, and uses a `:` as its seperator. A few key directives:
|
||||||
`host`, followed by a hostname signifies a host that will be available. Similar to nginx's `server_name` directive.
|
`host`, followed by a hostname signifies a host that will be available. Similar to nginx's `server_name` directive.
|
||||||
`globals` signifies all values that are of global importance, like the key and certificate file.
|
`globals` signifies all values that are of global importance, like the key and certificate file.
|
||||||
@@ -61,12 +74,15 @@ It uses a key-value syntax, and uses a `:` as its seperator. A few key directive
|
|||||||
`block-ua` signifies if a specific (or loosely matched) User-Agent must be blocked from accessing the site.
|
`block-ua` signifies if a specific (or loosely matched) User-Agent must be blocked from accessing the site.
|
||||||
`proxy` signifies if a the server needs to get the response from a different (remote) server but still needs to be available at this host.
|
`proxy` signifies if a the server needs to get the response from a different (remote) server but still needs to be available at this host.
|
||||||
`max-length` signifies the maximum length a request may have.
|
`max-length` signifies the maximum length a request may have.
|
||||||
|
`threading` signifies if the threading engine should be enabled. This will lend more performance, but should be disabled if you use scripts that are not thread-safe.
|
||||||
AmethystConf has only 4 datatypes: `String`, `Boolean`, `Function` and `None`. A quick rundown:
|
AmethystConf has only 4 datatypes: `String`, `Boolean`, `Function` and `None`. A quick rundown:
|
||||||
`String` is the everything datatype. Everything is assumed to be a `String` unless it falls under the other categories.
|
`String` is the everything datatype. Everything is assumed to be a `String` unless it falls under the other categories.
|
||||||
`Boolean` is the datatype used to enable/disable features. A `Boolean` can have one of two possible values: `1` or `0`.
|
`Boolean` is the datatype used to enable/disable features. A `Boolean` can have one of two possible values: `1` or `0`.
|
||||||
`Function` is the datatype used in `match()`, it signifies that the parser has to do some work on this string before it can use it.
|
`Function` is the datatype used in `match()`, it signifies that the parser has to do some work on this string before it can use it.
|
||||||
`None` is the datatype assigned to any key without a value.
|
`None` is the datatype assigned to any key without a value.
|
||||||
|
|
||||||
## PES (Python Extension Script)
|
## PES (Python Extension Script)
|
||||||
|
|
||||||
The PES (Python Extension Script) is one of Amethysts main selling points. It's a new type of a dynamic page. A PES file is pretty much a
|
The PES (Python Extension Script) is one of Amethysts main selling points. It's a new type of a dynamic page. A PES file is pretty much a
|
||||||
Python script with some conventions. Currently it is in very alpha form. It will be heavily improved upon to make sure even people with no
|
Python script with some conventions. Currently it is in very alpha form. It will be heavily improved upon to make sure even people with no
|
||||||
Python knowledge can work with it. Here's how it works:
|
Python knowledge can work with it. Here's how it works:
|
||||||
@@ -75,6 +91,7 @@ From there, the decoded request is given to you to play with. All of Amethysts r
|
|||||||
a function will be added to hand the request back to Amethyst, if it is deemed not suitable for PES mode.
|
a function will be added to hand the request back to Amethyst, if it is deemed not suitable for PES mode.
|
||||||
Once you're done manipulating the request, all you have to do is call `return self.build_response(http_status_code, resp_body, mimetype)` and Amethyst will handle the rest.
|
Once you're done manipulating the request, all you have to do is call `return self.build_response(http_status_code, resp_body, mimetype)` and Amethyst will handle the rest.
|
||||||
A few examples of what can be achieved with PES mode without writing any other language than Python, HTML and CSS:
|
A few examples of what can be achieved with PES mode without writing any other language than Python, HTML and CSS:
|
||||||
|
|
||||||
* Showing a random image from the `/pics` folder upon requesting `/randompic` from the server
|
* Showing a random image from the `/pics` folder upon requesting `/randompic` from the server
|
||||||
* Dynamically updating the time on a website
|
* Dynamically updating the time on a website
|
||||||
* Create a complex calculator
|
* Create a complex calculator
|
||||||
@@ -87,6 +104,7 @@ in browser, where it's static, as PES cannot change anything there.
|
|||||||
**WARNING!**
|
**WARNING!**
|
||||||
PES is an advanced feature! You can absolutely compromise the security of your webserver by having a misconfigured PES file. While Amethyst still has a few protection measures built-in
|
PES is an advanced feature! You can absolutely compromise the security of your webserver by having a misconfigured PES file. While Amethyst still has a few protection measures built-in
|
||||||
that activate before any request reaches the PES, but some are bypassed unless manually invoked in the PES. Because of that, here's a general user advisory:
|
that activate before any request reaches the PES, but some are bypassed unless manually invoked in the PES. Because of that, here's a general user advisory:
|
||||||
|
|
||||||
* Use `self.fh.read_file(file_path, host=None)` instead of `open(file_path)` because of file inclusion or directory traversal concerns.
|
* Use `self.fh.read_file(file_path, host=None)` instead of `open(file_path)` because of file inclusion or directory traversal concerns.
|
||||||
* Use `self.fh.write_file(file_path, host=None)` instead of `open(file_path)` because of file inclusion or directory traversal concerns.
|
* Use `self.fh.write_file(file_path, host=None)` instead of `open(file_path)` because of file inclusion or directory traversal concerns.
|
||||||
* **NEVER** allow the PES to run shell code!
|
* **NEVER** allow the PES to run shell code!
|
||||||
|
|||||||
+3
-3
@@ -2,9 +2,8 @@
|
|||||||
|
|
||||||
host * {
|
host * {
|
||||||
directory:./html
|
directory:./html
|
||||||
apimode:0
|
pesmode:0
|
||||||
block-ua:match("Discordbot")
|
index:index.html
|
||||||
index:index2.html
|
|
||||||
}
|
}
|
||||||
|
|
||||||
globals {
|
globals {
|
||||||
@@ -15,4 +14,5 @@ globals {
|
|||||||
key:./key.pem
|
key:./key.pem
|
||||||
cert:./cert.pem
|
cert:./cert.pem
|
||||||
max-length:8192
|
max-length:8192
|
||||||
|
threading:1
|
||||||
}
|
}
|
||||||
|
|||||||
+132
-90
@@ -61,7 +61,7 @@ except ImportError:
|
|||||||
)
|
)
|
||||||
# pass
|
# pass
|
||||||
|
|
||||||
AMETHYST_BUILD_NUMBER = "0083"
|
AMETHYST_BUILD_NUMBER = "0.3.0-0114-mt-tryout1"
|
||||||
AMETHYST_REPO = "https://git.novacow.ch/Nova/PyWebServer/"
|
AMETHYST_REPO = "https://git.novacow.ch/Nova/PyWebServer/"
|
||||||
|
|
||||||
|
|
||||||
@@ -143,7 +143,7 @@ class FileHandler:
|
|||||||
def read_file(self, file_path, directory=None):
|
def read_file(self, file_path, directory=None):
|
||||||
if "../" in file_path or "%" in file_path:
|
if "../" in file_path or "%" in file_path:
|
||||||
return 403, None
|
return 403, None
|
||||||
if file_path == "api.py":
|
if file_path == "pes.py":
|
||||||
return 404, None
|
return 404, None
|
||||||
|
|
||||||
if directory is not None:
|
if directory is not None:
|
||||||
@@ -192,7 +192,7 @@ class RequestParser:
|
|||||||
|
|
||||||
def extract_header(self, header: str, request: bytes | str):
|
def extract_header(self, header: str, request: bytes | str):
|
||||||
if isinstance(request, bytes):
|
if isinstance(request, bytes):
|
||||||
request = request.decode("utf-8", "ignore")
|
request = request.decode("iso-8859-1", "ignore")
|
||||||
lines = request.splitlines()
|
lines = request.splitlines()
|
||||||
for line in lines:
|
for line in lines:
|
||||||
if line.startswith(header):
|
if line.startswith(header):
|
||||||
@@ -200,13 +200,13 @@ class RequestParser:
|
|||||||
return value
|
return value
|
||||||
return None
|
return None
|
||||||
|
|
||||||
def parse_request_line(self, line, host):
|
def parse_request_line(self, line, host, no_mod=False):
|
||||||
"""Parses the HTTP request line."""
|
"""Parses the HTTP request line."""
|
||||||
try:
|
try:
|
||||||
method, path, version = line.split(" ")
|
method, path, version = line.split(" ")
|
||||||
except ValueError:
|
except ValueError:
|
||||||
return None, None, None
|
return None, None, None
|
||||||
if path.endswith("/") or ("." not in path):
|
if (path.endswith("/") or ("." not in path)) and (not no_mod):
|
||||||
if not path.endswith("/"):
|
if not path.endswith("/"):
|
||||||
path += "/"
|
path += "/"
|
||||||
index = self.file_handler.read_config("index", host) or "index.html"
|
index = self.file_handler.read_config("index", host) or "index.html"
|
||||||
@@ -276,6 +276,7 @@ class RequestParser:
|
|||||||
class ProxyServer:
|
class ProxyServer:
|
||||||
def __init__(self, fh):
|
def __init__(self, fh):
|
||||||
self.file_handler: FileHandler = fh
|
self.file_handler: FileHandler = fh
|
||||||
|
self.rq: RequestParser = RequestParser()
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def recv_all(sock):
|
def recv_all(sock):
|
||||||
@@ -293,16 +294,24 @@ class ProxyServer:
|
|||||||
def try_connection(
|
def try_connection(
|
||||||
self, host: str, port: int, data: bytes, chost: str, force_tls: bool = None
|
self, host: str, port: int, data: bytes, chost: str, force_tls: bool = None
|
||||||
):
|
):
|
||||||
if port in [443, 8443, 9443]:
|
|
||||||
do_tls = True
|
|
||||||
else:
|
|
||||||
if force_tls is True:
|
|
||||||
do_tls = True
|
|
||||||
else:
|
|
||||||
do_tls = False
|
|
||||||
print(f"\n\n\nchost: {chost}\n\n\n")
|
print(f"\n\n\nchost: {chost}\n\n\n")
|
||||||
nhost = self.file_handler.read_config("proxy", chost)
|
nhost = self.file_handler.read_config("proxy", chost)
|
||||||
print(f"\n\n\nnhost: {nhost}\n\n\n")
|
print(f"\n\n\nnhost: {nhost}\n\n\n")
|
||||||
|
# nhost will include http or https.
|
||||||
|
if nhost.startswith("https"):
|
||||||
|
nhost = nhost[6:-1]
|
||||||
|
do_tls = True
|
||||||
|
elif nhost.startswith("http"):
|
||||||
|
nhost = nhost[5:-1]
|
||||||
|
do_tls = False
|
||||||
|
else:
|
||||||
|
raise SyntaxError(
|
||||||
|
"Syntax error in config! Key: `proxy` Reason: `Expected http([...]) or https([...]), not "
|
||||||
|
f"{nhost[:6]}[...]{nhost[-1:]}!`"
|
||||||
|
)
|
||||||
|
if force_tls is True:
|
||||||
|
do_tls = True
|
||||||
|
print(f"\n\n\nnhost: {nhost}\n\n\n")
|
||||||
if ":" in nhost:
|
if ":" in nhost:
|
||||||
nport = int(nhost.split(":")[1])
|
nport = int(nhost.split(":")[1])
|
||||||
nhost = nhost.split(":")[0]
|
nhost = nhost.split(":")[0]
|
||||||
@@ -311,47 +320,52 @@ class ProxyServer:
|
|||||||
print(f"{nhost}, {nport}, {data}")
|
print(f"{nhost}, {nport}, {data}")
|
||||||
data = self.reset_host(nhost, nport, data)
|
data = self.reset_host(nhost, nport, data)
|
||||||
try:
|
try:
|
||||||
return self.tcp_send(host, port, data, do_tls)
|
print("Waiting on TCP start.")
|
||||||
except Exception:
|
return self.tcp_send(nhost, nport, data, do_tls)
|
||||||
if do_tls is False:
|
except Exception as e:
|
||||||
print("Retrying with TLS...")
|
raise Exception(f"Server replied unexpected. Reply from Python subsystem: {e}")
|
||||||
return self.try_connection(host, port, data, chost, True)
|
|
||||||
else:
|
|
||||||
raise
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def reset_host(host: str, port: int, data: bytes):
|
def reset_host(host: str, port: int, data: bytes):
|
||||||
data = data.decode()
|
header_end = data.find(b"\r\n\r\n")
|
||||||
data = data.splitlines()
|
|
||||||
for line in data:
|
if header_end == -1:
|
||||||
print(line)
|
return data
|
||||||
if line.startswith("Host:"):
|
|
||||||
|
header_bytes = data[:header_end]
|
||||||
|
body = data[header_end + 4:]
|
||||||
|
|
||||||
|
headers = header_bytes.decode("iso-8859-1")
|
||||||
|
|
||||||
|
lines = headers.split("\r\n")
|
||||||
|
new_lines = []
|
||||||
|
|
||||||
|
for line in lines:
|
||||||
|
lower = line.lower()
|
||||||
|
|
||||||
|
if lower.startswith("host:"):
|
||||||
if port not in [80, 443]:
|
if port not in [80, 443]:
|
||||||
new_line = f"Host: {host}:{port}"
|
line = f"Host: {host}:{port}"
|
||||||
else:
|
else:
|
||||||
new_line = f"Host: {host}"
|
line = f"Host: {host}"
|
||||||
idx = data.index(line)
|
|
||||||
data[idx] = new_line
|
elif lower.startswith("connection:"):
|
||||||
print(f"\n\n\n{idx}\n\n\n")
|
line = "Connection: close"
|
||||||
if line.startswith("Connection:"):
|
|
||||||
idx = data.index(line)
|
new_lines.append(line)
|
||||||
new_line = "Connection: close"
|
|
||||||
data[idx] = new_line
|
rebuilt_headers = "\r\n".join(new_lines).encode("iso-8859-1")
|
||||||
data = "\r\n".join(data)
|
|
||||||
data = f"{data}\r\n\r\n"
|
return rebuilt_headers + b"\r\n\r\n" + body
|
||||||
print(data)
|
|
||||||
return data.encode()
|
|
||||||
# return data
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def create_tls_context():
|
def create_tls_context():
|
||||||
# Create a context that by default verifies with system CAs
|
|
||||||
ctx = ssl.create_default_context()
|
ctx = ssl.create_default_context()
|
||||||
ctx.check_hostname = False
|
ctx.check_hostname = False
|
||||||
ctx.verify_mode = ssl.CERT_NONE
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
return ctx
|
return ctx
|
||||||
|
|
||||||
def tcp_send(self, host, port, data: bytes, do_tls: bool):
|
def tcp_send(self, host, port, data: bytes, do_tls: booll):
|
||||||
try:
|
try:
|
||||||
with socket.create_connection((host, port), timeout=10) as raw_sock:
|
with socket.create_connection((host, port), timeout=10) as raw_sock:
|
||||||
raw_sock.settimeout(10)
|
raw_sock.settimeout(10)
|
||||||
@@ -363,11 +377,35 @@ class ProxyServer:
|
|||||||
) as ssock:
|
) as ssock:
|
||||||
ssock.sendall(data)
|
ssock.sendall(data)
|
||||||
print("data reached")
|
print("data reached")
|
||||||
return self.recv_all(ssock)
|
resp = self.recv_all(ssock)
|
||||||
|
if self.rq.extract_header("Transfer-Encoding", resp) == "chunked":
|
||||||
|
ssock.sendall(b"TRANSER-ENCODING IS NOT SUPPORTED")
|
||||||
|
ssock.close()
|
||||||
|
resp = (
|
||||||
|
"HTTP/1.1 502 Gateway Error\r\nConnection: close\r\nContent-Length: 270\r\n"
|
||||||
|
f"Server: Amethyst/build-{AMETHYST_BUILD_NUMBER}\r\n\r\n"
|
||||||
|
"Gateway Error.\nThe upstream server tried to use a a transfer mode not "
|
||||||
|
"yet supported\nExact error:\nE_DATA_STREAMING_NOT_SUPPORTED\n"
|
||||||
|
"The unsupported mode:\nTransfer-Encoding: chunked.\n"
|
||||||
|
"There is no fix. The problem lies with the proxy, and is not a fault of the upstream server."
|
||||||
|
)
|
||||||
|
return resp
|
||||||
else:
|
else:
|
||||||
|
print(f"\n\n\nraw data: {data}\n\n\n")
|
||||||
raw_sock.sendall(data)
|
raw_sock.sendall(data)
|
||||||
|
print("Waiting for response...")
|
||||||
resp = self.recv_all(raw_sock)
|
resp = self.recv_all(raw_sock)
|
||||||
print(f"resp = {resp}")
|
if self.rq.extract_header("Transfer-Encoding", resp) is not None:
|
||||||
|
raw_sock.sendall(b"TRANSER-ENCODING IS NOT SUPPORTED")
|
||||||
|
raw_sock.close()
|
||||||
|
resp = (
|
||||||
|
"HTTP/1.1 502 Gateway Error\r\nConnection: close\r\nContent-Length: 270\r\n"
|
||||||
|
f"Server: Amethyst/build-{AMETHYST_BUILD_NUMBER}\r\n\r\n"
|
||||||
|
"Gateway Error.\nThe upstream server tried to use a a transfer mode not "
|
||||||
|
"yet supported\nExact error:\nE_DATA_STREAMING_NOT_SUPPORTED\n"
|
||||||
|
"The unsupported mode:\nTransfer-Encoding: chunked.\n"
|
||||||
|
"There is no fix. The problem lies with the proxy, and is not a fault of the upstream server."
|
||||||
|
)
|
||||||
return resp
|
return resp
|
||||||
except Exception:
|
except Exception:
|
||||||
raise
|
raise
|
||||||
@@ -412,9 +450,11 @@ class WebServer:
|
|||||||
|
|
||||||
self.http_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
|
self.http_socket = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
|
||||||
self.http_socket.bind(("::", self.http_port))
|
self.http_socket.bind(("::", self.http_port))
|
||||||
|
self.http_socket.settimeout(25)
|
||||||
|
|
||||||
self.https_socket_raw = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
|
self.https_socket_raw = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
|
||||||
self.https_socket_raw.bind(("::", self.https_port))
|
self.https_socket_raw.bind(("::", self.https_port))
|
||||||
|
self.https_socket_raw.settimeout(25)
|
||||||
|
|
||||||
self.proxy_handler = ProxyServer(self.file_handler)
|
self.proxy_handler = ProxyServer(self.file_handler)
|
||||||
|
|
||||||
@@ -447,40 +487,39 @@ class WebServer:
|
|||||||
self.running = True
|
self.running = True
|
||||||
|
|
||||||
def start(self, http, https):
|
def start(self, http, https):
|
||||||
signal.signal(signal.SIGINT, self.shutdown)
|
|
||||||
signal.signal(signal.SIGTERM, self.shutdown)
|
|
||||||
|
|
||||||
http_thread = threading.Thread(target=self.start_http, daemon=True)
|
|
||||||
https_thread = threading.Thread(target=self.start_https, daemon=True)
|
|
||||||
|
|
||||||
if https is True:
|
if https is True:
|
||||||
if self.skip_ssl is True:
|
if self.skip_ssl is True:
|
||||||
print("WARN: You have enabled HTTPS without SSL!!")
|
print("WARN: You have enabled HTTPS without SSL!!")
|
||||||
yn = input("Is this intended behaviour? [y/N] ")
|
yn = input("Is this intended behaviour? [y/N] ")
|
||||||
if yn.lower() == "n":
|
if yn.lower() == "n":
|
||||||
exit(1)
|
exit(1)
|
||||||
https_thread.start()
|
self.start_https()
|
||||||
else:
|
else:
|
||||||
self.https_socket.close()
|
self.https_socket.close()
|
||||||
if http is True:
|
if http is True:
|
||||||
http_thread.start()
|
self.start_http()
|
||||||
else:
|
else:
|
||||||
self.http_socket.close()
|
self.http_socket.close()
|
||||||
|
|
||||||
http_thread.join()
|
|
||||||
https_thread.join()
|
|
||||||
|
|
||||||
def start_http(self):
|
def start_http(self):
|
||||||
self.http_socket.listen(5)
|
self.http_socket.listen(5)
|
||||||
print(f"HTTP server listening on port {self.http_port}...")
|
print(f"HTTP server listening on port {self.http_port}...")
|
||||||
while self.running:
|
while self.running:
|
||||||
try:
|
try:
|
||||||
conn, addr = self.http_socket.accept()
|
conn, addr = self.http_socket.accept()
|
||||||
self.handle_connection(conn, addr)
|
if self.file_handler.read_config("threading") is True:
|
||||||
except Exception as e:
|
threading.Thread(
|
||||||
print(f"HTTP error: {e}")
|
target=self.handle_connection,
|
||||||
|
args=(conn, addr),
|
||||||
|
daemon=True
|
||||||
|
).start()
|
||||||
|
else:
|
||||||
|
self.handle_connection(conn, addr)
|
||||||
except OSError:
|
except OSError:
|
||||||
break
|
break
|
||||||
|
except Exception as e:
|
||||||
|
if not "timeout" in f"{e}":
|
||||||
|
print(f"HTTP error: {e}")
|
||||||
|
|
||||||
def start_https(self):
|
def start_https(self):
|
||||||
self.https_socket.listen(5)
|
self.https_socket.listen(5)
|
||||||
@@ -488,13 +527,19 @@ class WebServer:
|
|||||||
while self.running:
|
while self.running:
|
||||||
try:
|
try:
|
||||||
conn, addr = self.https_socket.accept()
|
conn, addr = self.https_socket.accept()
|
||||||
self.handle_connection(conn, addr)
|
if self.file_handler.read_config("threading") is True:
|
||||||
except Exception as e:
|
threading.Thread(
|
||||||
print(
|
target=self.handle_connection,
|
||||||
f"HTTPS error: {e}"
|
args=(conn, addr),
|
||||||
) # be ready for ssl errors if you use a self-sign!!
|
daemon=True
|
||||||
|
).start()
|
||||||
|
else:
|
||||||
|
self.handle_connection(conn, addr)
|
||||||
except OSError:
|
except OSError:
|
||||||
break
|
break
|
||||||
|
except Exception as e:
|
||||||
|
if not "timeout" in f"{e}":
|
||||||
|
print(f"HTTPS error: {e}")
|
||||||
|
|
||||||
def handle_connection(self, conn, addr):
|
def handle_connection(self, conn, addr):
|
||||||
try:
|
try:
|
||||||
@@ -528,22 +573,7 @@ class WebServer:
|
|||||||
body += chunk
|
body += chunk
|
||||||
|
|
||||||
data += body
|
data += body
|
||||||
#
|
request = data.decode("iso-8859-1", errors="ignore")
|
||||||
# print(f"\n\nbody length {len(body)}\n\n")
|
|
||||||
# print("headers len", len(headers))
|
|
||||||
# print("rest len", len(rest))
|
|
||||||
# print("body len", len(body))
|
|
||||||
# print("content_length", content_length)
|
|
||||||
#
|
|
||||||
# print("last 200 bytes of body:")
|
|
||||||
# print(repr(body[-200:]))
|
|
||||||
# print("body starts with:")
|
|
||||||
# print(repr(body[:100]))
|
|
||||||
# print(f"body: {body}")
|
|
||||||
print(b"data: " + headers + b"\r\n\r\n" + body)
|
|
||||||
# # data = conn.recv(32768)
|
|
||||||
# print(f"len(data) = {len(data)}")
|
|
||||||
request = data.decode(errors="ignore")
|
|
||||||
if not data:
|
if not data:
|
||||||
response = self.build_response(
|
response = self.build_response(
|
||||||
400, "Bad Request"
|
400, "Bad Request"
|
||||||
@@ -617,14 +647,14 @@ class WebServer:
|
|||||||
value = self.file_handler.read_config("proxy", host)
|
value = self.file_handler.read_config("proxy", host)
|
||||||
if ":" in value:
|
if ":" in value:
|
||||||
host = value.split(":")[0]
|
host = value.split(":")[0]
|
||||||
port = int(value.split(":")[1])
|
port = int(value.split(":")[1][:-1])
|
||||||
else:
|
else:
|
||||||
host = value
|
host = value
|
||||||
port = 443
|
port = 443
|
||||||
return self.proxy_handler.try_connection(
|
return self.proxy_handler.try_connection(
|
||||||
host,
|
host,
|
||||||
port,
|
port,
|
||||||
data.encode(),
|
data.encode("iso-8859-1"),
|
||||||
orig_host,
|
orig_host,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -643,13 +673,25 @@ class WebServer:
|
|||||||
or self.file_handler.base_dir
|
or self.file_handler.base_dir
|
||||||
)
|
)
|
||||||
|
|
||||||
if self.file_handler.read_config("apimode", host) is True:
|
if bool(self.file_handler.read_config("pesmode", host)) is True:
|
||||||
if not os.path.join(os.getcwd(), directory) in sys.path:
|
if not os.path.join(os.getcwd(), directory) in sys.path:
|
||||||
sys.path.append(f"{os.path.join(os.getcwd(), directory)}")
|
sys.path.append(f"{os.path.join(os.getcwd(), directory)}")
|
||||||
import api
|
import pes
|
||||||
|
try:
|
||||||
|
pesclass = pes.PES()
|
||||||
|
threadcompat = pesclass.THREAD_SAFETY
|
||||||
|
# if not threadcompat:
|
||||||
|
return pesclass.on_request(data)
|
||||||
|
except Exception as e:
|
||||||
|
return self.build_response(
|
||||||
|
500,
|
||||||
|
"Amethyst is currently unable to serve your request. Below is debug info.\r\n"
|
||||||
|
f"Error: {e}; Version: amethyst-b{AMETHYST_BUILD_NUMBER}\r\n"
|
||||||
|
"You cannot do anything at this time, the server owner has made a misconfiguration in their Python Extension Script",
|
||||||
|
)
|
||||||
|
|
||||||
apiclass = api.API()
|
if bool(self.file_handler.read_config("621mode", host)) is True:
|
||||||
return apiclass.on_request(data)
|
return self.build_response(621, "")
|
||||||
|
|
||||||
file_content, mimetype = self.file_handler.read_file(path, directory)
|
file_content, mimetype = self.file_handler.read_file(path, directory)
|
||||||
|
|
||||||
@@ -749,14 +791,14 @@ class WebServer:
|
|||||||
|
|
||||||
if status_code == 621:
|
if status_code == 621:
|
||||||
headers = (
|
headers = (
|
||||||
f"HTTP/1.1 {status_code} {status_message}\r\n"
|
"HTTP/1.1 302 UwU Nya!\r\n"
|
||||||
"Server: Amethyst/build-0621\r\n"
|
"Server: Amethyst/build-0621\r\n"
|
||||||
"Content-Length: 30\r\n"
|
"Content-Length: 0\r\n"
|
||||||
f"Connection: close\r\n\r\n"
|
"Connection: close\r\n"
|
||||||
)
|
"Note: congrats, you found a funny. i guess.\r\n"
|
||||||
body = "https://e621.net/posts/6155664"
|
"Host: https://e621.net/posts/\r\n\r\n"
|
||||||
|
).encode("iso-8859-1")
|
||||||
print(f"{headers + body}")
|
body = "".encode("iso-8859-1")
|
||||||
return headers + body
|
return headers + body
|
||||||
|
|
||||||
def shutdown(self, signum, frame):
|
def shutdown(self, signum, frame):
|
||||||
|
|||||||
+38
-20
@@ -6,28 +6,50 @@ import datetime
|
|||||||
|
|
||||||
|
|
||||||
class AutoCertGen:
|
class AutoCertGen:
|
||||||
def __init__(self):
|
def __init__(self, name="website", org="organization", locale="place", province="province", country="ZZ", dns="localhost"):
|
||||||
pass
|
self.name = name
|
||||||
|
self.org = org
|
||||||
|
self.locale = locale
|
||||||
|
self.province = province
|
||||||
|
self.country = country
|
||||||
|
self.dns = dns
|
||||||
|
|
||||||
def gen_cert(self):
|
def generate_self_signed_cert(self, different_issuer=False):
|
||||||
# Generate private key
|
|
||||||
private_key = rsa.generate_private_key(
|
private_key = rsa.generate_private_key(
|
||||||
public_exponent=65537,
|
public_exponent=65537,
|
||||||
key_size=2048,
|
key_size=2048,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Define subject and issuer (self-signed)
|
if different_issuer is True:
|
||||||
subject = issuer = x509.Name(
|
subject = x509.Name(
|
||||||
[
|
[
|
||||||
x509.NameAttribute(NameOID.COUNTRY_NAME, "ZZ"),
|
x509.NameAttribute(NameOID.COUNTRY_NAME, self.country),
|
||||||
x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, "Some province"),
|
x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, self.province),
|
||||||
x509.NameAttribute(NameOID.LOCALITY_NAME, "Some place"),
|
x509.NameAttribute(NameOID.LOCALITY_NAME, self.locale),
|
||||||
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "Some org"),
|
x509.NameAttribute(NameOID.ORGANIZATION_NAME, self.org),
|
||||||
x509.NameAttribute(NameOID.COMMON_NAME, "localhost"),
|
x509.NameAttribute(NameOID.COMMON_NAME, self.name),
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
issuer = x509.Name(
|
||||||
|
[
|
||||||
|
x509.NameAttribute(NameOID.COUNTRY_NAME, "RU"),
|
||||||
|
x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, "Красноярск Область"),
|
||||||
|
x509.NameAttribute(NameOID.LOCALITY_NAME, "Красноярск"),
|
||||||
|
x509.NameAttribute(NameOID.ORGANIZATION_NAME, "Amethyst Group"),
|
||||||
|
x509.NameAttribute(NameOID.COMMON_NAME, "Amethyst Untrusted Signing Certificate"),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
subject = issuer = x509.Name(
|
||||||
|
[
|
||||||
|
x509.NameAttribute(NameOID.COUNTRY_NAME, self.country),
|
||||||
|
x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, self.province),
|
||||||
|
x509.NameAttribute(NameOID.LOCALITY_NAME, self.locale),
|
||||||
|
x509.NameAttribute(NameOID.ORGANIZATION_NAME, self.org),
|
||||||
|
x509.NameAttribute(NameOID.COMMON_NAME, self.name),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
# Create certificate
|
|
||||||
certificate = (
|
certificate = (
|
||||||
x509.CertificateBuilder()
|
x509.CertificateBuilder()
|
||||||
.subject_name(subject)
|
.subject_name(subject)
|
||||||
@@ -39,12 +61,11 @@ class AutoCertGen:
|
|||||||
datetime.datetime.utcnow() + datetime.timedelta(days=365)
|
datetime.datetime.utcnow() + datetime.timedelta(days=365)
|
||||||
) # 1 year validity
|
) # 1 year validity
|
||||||
.add_extension(
|
.add_extension(
|
||||||
x509.SubjectAlternativeName([x509.DNSName("localhost")]), critical=False
|
x509.SubjectAlternativeName([x509.DNSName(self.dns)]), critical=False
|
||||||
)
|
)
|
||||||
.sign(private_key, hashes.SHA256())
|
.sign(private_key, hashes.SHA256())
|
||||||
)
|
)
|
||||||
|
|
||||||
# Save private key
|
|
||||||
with open("key.pem", "wb") as f:
|
with open("key.pem", "wb") as f:
|
||||||
f.write(
|
f.write(
|
||||||
private_key.private_bytes(
|
private_key.private_bytes(
|
||||||
@@ -54,8 +75,5 @@ class AutoCertGen:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
# Save certificate
|
|
||||||
with open("cert.pem", "wb") as f:
|
with open("cert.pem", "wb") as f:
|
||||||
f.write(certificate.public_bytes(serialization.Encoding.PEM))
|
f.write(certificate.public_bytes(serialization.Encoding.PEM))
|
||||||
|
|
||||||
print("Self-signed certificate and private key generated for HTTPS server!")
|
|
||||||
|
|||||||
+1
-1
@@ -8,7 +8,7 @@
|
|||||||
<h1>Hello from Amethyst!</h1>
|
<h1>Hello from Amethyst!</h1>
|
||||||
<h2>This page confirms Amethyst can read files from your PC or server and serve them to your browser!</h2>
|
<h2>This page confirms Amethyst can read files from your PC or server and serve them to your browser!</h2>
|
||||||
<p>This is a test page, if you aren't the server owner, they might not have finished setting up their site, be patient. If this doesn't go away after a while, tell them they've made an oopsie</p>
|
<p>This is a test page, if you aren't the server owner, they might not have finished setting up their site, be patient. If this doesn't go away after a while, tell them they've made an oopsie</p>
|
||||||
<p>This server runs Amethyst build 0080</p>
|
<p>This server runs Amethyst build 0.3.0-0114-mt-tryout1</p>
|
||||||
</center>
|
</center>
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|||||||
@@ -9,10 +9,10 @@ import os
|
|||||||
|
|
||||||
if not os.getcwd() in sys.path:
|
if not os.getcwd() in sys.path:
|
||||||
sys.path.append(os.getcwd())
|
sys.path.append(os.getcwd())
|
||||||
import pywebsrv
|
import amethyst
|
||||||
|
|
||||||
|
|
||||||
class API:
|
class PES:
|
||||||
"""
|
"""
|
||||||
class
|
class
|
||||||
"""
|
"""
|
||||||
@@ -20,7 +20,10 @@ class API:
|
|||||||
def __init__(self):
|
def __init__(self):
|
||||||
# DO NOT USE THIS CLASS FOR PROGRAM, ONLY ON_REQUEST PLEASE!!
|
# DO NOT USE THIS CLASS FOR PROGRAM, ONLY ON_REQUEST PLEASE!!
|
||||||
# Below go definitions to get things working.
|
# Below go definitions to get things working.
|
||||||
self.build_response = pywebsrv.WebServer.build_binary_response
|
self.build_response = amethyst.WebServer.build_binary_response
|
||||||
|
self.fh = amethyst.FileHandler("..")
|
||||||
|
self.rq = amethyst.RequestParser()
|
||||||
|
self.THREAD_SAFETY: bool = True
|
||||||
|
|
||||||
def on_request(self, req):
|
def on_request(self, req):
|
||||||
return self.build_response(200, "This is a test", "text/html")
|
return self.build_response(200, b"Heyhey! This is the default PES script!", "text/html")
|
||||||
Reference in New Issue
Block a user