[Mod] Auth Redux v2.14 (Authentication Handler) [auth_rx]

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

[Mod] Auth Redux v2.14 (Authentication Handler) [auth_rx]

by sorcerykid » Post

Image

Auth Redux Mod v2.14
auth_rx (by sorcerykid)

Auth Redux is a drop-in replacement for the builtin authentication handler of Minetest. It is designed from the ground up to be robust and secure enough for use on high-traffic Minetest servers, while also addressing a number of outstanding engine bugs (including #5334 and #6783 and #4451) as described last year in the Critical bug in builtin authentication handler forum topic.
  • Image
Auth Redux is completely Lua-driven, with no external dependencies. It interfaces seamlessly with Minetest's authentication handler API, therefore no complicated setup or configuration is required. It is intended to be compatible with all versions of Minetest 0.4.14+, providing server administrators with both flexible and fine-grained access control.

Recommended Enhancement: The DataMiner Mod extends the capabilities of Auth Redux, providing comprehensive player and server statistics within a graphical user-interface, in addition to an API for custom-tailored log analysis and reporting directly from the command line

Repository:

https://bitbucket.org/sorcerykid/auth_rx

https://gitlab.com/sorcerykid/auth_rx (mirror)
https://notabug.org/sorcerykid/auth_rx (mirror)

Download Archive (.zip)
Download Archive (.tar.gz)

Dependencies:

ActiveFormspecs Mod (optional)

Source Code License:

The MIT License (MIT)

Installation:

Only a few basic steps are required to begin using the Auth_Redux mod immediately with your server.
  1. Unzip the archive into the mods directory of your game
  2. Rename the auth_rx-master directory to "auth_rx"
  3. Execute the conversion scripts (see usage instructions below)
Linux distributions already include the awk command-line utility, so no additional setup is required. However Microsoft Windows users will need to install Gawk for Windows.

Documentation:
  1. Introduction
  2. Basic Database Import
  3. Advanced Database Import
  4. Working with Rulesets
  5. Command-Line Analytics Script
  6. Database Management Console
  7. Additional Callback Support
  8. Migrating to Auth Redux Lite
  9. Technical Information
Overview:

Auth Redux provides an efficient and scalable back-end solution for player authentication, making it suitable for even the most demanding server environments. It has been put to the test with a database of over 520,000 registered users -- with no in-game overhead!
  • Image

    High-Performance Journaling System
    To protect the master database against corruption and data loss, all transactions are serially recorded within a journal, thereby reducing the frequency and intensity of disk writes. In the event of a server crash or power outage, any pending transactions are committed to the master database.

    Stateful Rule-Based Login Filtering
    MARS is an easy-to-learn scripting language that affords a first line of defense against unwanted visitors. Both restrictive and permissive security models can be enforced via rulesets, allowing every login request to be filtered according to conditional action-chains with custom error messages.

    Expanded Master Database Schema
    The master database supports additional metadata to aid in reviewing and monitoring login activity, including the timestamp of the first login, the total time spent playing, the total number of player sessions, etc. This information can prove invaluable for command-line reporting and analysis tools.
The project is a WIP and will be undergoing continuous development based upon your suggestions and feedback. New features and bug-fixes will be announced here as they become available. Full documentation is mirrored on GitLab: https://gitlab.com/snippets/1750802

Please notify me if you encounter technical problems, such as errors or incompatibilities. Bugs should be reported to the issue tracker.

Auth Redux vs SQLite3

Recently, I decided to benchmark the performance of Auth Redux in order to weed out any potential bottlenecks. So I put it to the test against two different SQLite3 implementations using a database of over 522,000 records from the JT2 server.

Much to my surprise, Auth Redux doesn't suffer at all when it comes to raw speed. It's insanely fast compared to both sauth and the new Minetest 5.0 builtin. For every test, it vastly out-performed these other authentication handlers. I'm talking orders of magnitude in the hundreds and even thousands, as shown by these baseline measurements:
  • Image
Even the database file size of Auth Redux is just 2/3rds that of auth.sqlite used by Minetest 5.0. And yet the schema is far more extensive (including total time spent playing, initial login timestamp, total player sessions, etc.) When it comes to rsyncing files in the hundreds of megabytes over SSH, this can make a huge difference.

Image

Perhaps, the biggest area of concern for most admins is the execution time during player joins. If a high-traffic server averages around 30 players joining every minute at peak hour, sauth would consume over 20 seconds of CPU cycles (that is 1/3 of server time spent processing authorization). Meanwhile, auth_rx clocks in at just 3.8 milliseconds after 30 player joins.

For each test below, I performed five independent trials and averaged the results to get the most accurate figure possible.

Image

Image

Image

As these benchmarks reveal, there is no discernible benefit to using an SQLite3 backend for authentication over a flat-file database, whether in terms of reliability or performance. If anything, SQLite3 databases are a hindrance when it comes to random access of large volumes of data, and more than likely the culprit for why some high-traffic servers lately are experiencing notable lag spikes.
Last edited by sorcerykid on Sun Feb 17, 2019 00:58, edited 14 times in total.

twoelk
Member
Posts: 1482
Joined: Fri Apr 19, 2013 16:19
GitHub: twoelk
IRC: twoelk
In-game: twoelk
Location: northern Germany

Re: [Mod] Auth Redux v2.1 (Authentication Handler) [auth_rx]

by twoelk » Post

nice to see your still alive and kicking

looks interesting
can this also pass information to the spawning process?
like setting different spawns for different name patterns or privs or some ip ranges or some other identifier?

usecases could be to send newbies to another place than veterans or have school classes connecting over a restricted set of ip numbers sent to their individual ingame class project site.

the idea would be to use the sorting abilities of these rulesets and not having to trigger a second sorting on join.

User avatar
maikerumine
Member
Posts: 1420
Joined: Mon Aug 04, 2014 14:27
GitHub: maikerumine
In-game: maikerumine

Re: [Mod] Auth Redux v2.1 (Authentication Handler) [auth_rx]

by maikerumine » Post

So glad to see you active again, SK!

Quite impressive coding there, most impressive.
Talamh Survival Minetest-->viewtopic.php?f=10&t=12959

User avatar
Linuxdirk
Member
Posts: 3216
Joined: Wed Sep 17, 2014 11:21
In-game: Linuxdirk
Location: Germany
Contact:

Re: [Mod] Auth Redux v2.1 (Authentication Handler) [auth_rx]

by Linuxdirk » Post

Woah! I wonder why this is not in Minetest from the beginning on! Great work!

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.1 (Authentication Handler) [auth_rx]

by sorcerykid » Post

maikerumine wrote:So glad to see you active again, SK!
Quite impressive coding there, most impressive.
WOW, maike! How have you been? I've missed seeing you around!

Thanks so much for the words of encouragement. You are always so kind. I think you were the first person I PM'd when I was troubleshooting this bug on JT2 back in 2016. And you helped me a lot even back then. How time flies :)

I'm just happy to finally start releasing more of my mods (besides Wuzzy was always prodding me to be more "open" with my source code). And I know this is an issue that has plagued a lot of servers, esp. since many are still running older versions of Minetest, like 0.4.16 and even 0.4.15. So backwards compatibility seemed pretty crucial. I hope other server operators find it useful as well.

Congrats on your new ESM5 server. Take care dear friend!

~Leslie

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.1 (Authentication Handler) [auth_rx]

by sorcerykid » Post

twoelk wrote:nice to see your still alive and kicking
looks interesting
can this also pass information to the spawning process?
Hello, and thanks twoelk. Glad to see you are doing well too!

If you're asking about the rulesets, those were only intended for pre-authorization. I wanted to offer a simple and secure way for server operators to do basic access control on a per-world basis, without having to modify the game.

In theory, it should be feasible for a mod to obtain the filter results through a callback hook, perhaps something like minetest.register_on_authfilter( ). I'd been considering the addition of a "set" statement for custom ruleset variables. That might tie everything together nicely, from what you described. I'll have to give this some more thought :)

On a sidenote, I'd be more than glad to release my doors mod! Curiously, one the core devs said I won't. But, hey that's gossip for you xD Should you ever need something, please ask me personally.
Linuxdirk wrote:Woah! I wonder why this is not in Minetest from the beginning on! Great work!
Much appreciated! Between the weeks of coding and testing (plus my annoying perfectionism), it's nice to have it released at long last :D

The main impetus for this project was to have a better way to track and report player login activity. That's why I designed the journal from the ground up not only to protect against data loss and corruption, but to serve as a continuous security audit trail that could be analyzed using virtually any text-processing utility like awk, grep, Python, or Perl.

The decision to incorporate rulesets evolved from that, due to there being no mechanism outside of Lua for server operators to filter login requests conditionally. After days of searching Google for a Lua ruleset interpreter (to no avail), I decided that the only solution was to embed a specialized scripting language into the authentication handler.

User avatar
ManElevation
Member
Posts: 896
Joined: Tue Aug 02, 2016 22:04
GitHub: ManElevation
IRC: ManElevation
In-game: ManElevation
Location: Madrid,Spain

Re: [Mod] Auth Redux v2.1 (Authentication Handler) [auth_rx]

by ManElevation » Post

Wow you did spend ur time here. really nice coding.
+100
My Public Mods! Discord: Rottweiler Games#3368

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.1 (Authentication Handler) [auth_rx]

by sorcerykid » Post

New Database Import Script!

I revamped the database import script to further simplify installation. Now there are only two steps to begin using the Auth Redux mod:

Code: Select all

cd auth_rx
awk -f convert.awk -v mode=install <path_to_world>/auth.txt
Be sure to replace <path_to_world> with the world directory for your server (where the auth.txt file is located). That's it!

The script will import the database and create the journal and ruleset files automatically. If you want to preserve the ruleset or journal files from an existing installation, then set mode to "convert" instead of "install".

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.1 (Authentication Handler) [auth_rx]

by sorcerykid » Post

Version 2.2b Released

A new version of Auth Redux is ready for download. Here is a complete change log:
  • Added install option to database import script
  • Improved exception handling by AuthFilter class
  • Fixed parsing of number literals in rulesets
  • Fixed type-checking of try statements in rulesets
  • Included mod.conf and description.txt files
This release primarily consists of minor bug fixes, and of course the revamped database import script mentioned above.

In addition, the ruleset parser is much more lenient about syntax errors. Whereas exceptions used to always be fatal, they are now handled gracefully. A detailed error message is posted to the server's debug log and the end-user is notified "The server has experienced an internal error." For security reasons, I decided that an errant ruleset should probably be as restrictive as possible, although I might provide an option in minetest.conf to fine-tune this behavior.

User avatar
Lejo
Member
Posts: 718
Joined: Mon Oct 19, 2015 16:32
GitHub: Lejo1
In-game: Lejo

Re: [Mod] Auth Redux v2.2b (Authentication Handler) [auth_rx

by Lejo » Post

Very Great!!!!
Now the setting of the privs is 10000x faster.
Should be default!!!!

User avatar
Stix
Member
Posts: 1385
Joined: Fri Aug 04, 2017 14:19
IRC: nil
In-game: Stix [+alts]
Location: USA

Re: [Mod] Auth Redux v2.2b (Authentication Handler) [auth_rx

by Stix » Post

@sorcerkid Ah your back! Maybe while your here you could tell us if JT2 will be coming back? :-)

*Hope this isnt too much offtopic.
Hey, what can i say? I'm the bad guy.

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.2b (Authentication Handler) [auth_rx

by sorcerykid » Post

Version 2.3b Released

A new version of Auth Redux is ready for download. Here is a complete change log:
  • General code cleanup of AuthFilter class
  • Moved datasets into separate directory of world
  • Added two more comparison operators for rulesets
  • Tweaked pattern matching behavior in rulesets
  • Changed database search method to use Lua regexes
  • Removed hard-coded file names from database methods
Aside from some basic housekeeping under-the-hood, I introduced some new functionality as well. First and foremost, four additional wildcards are provided for even more precise pattern matching:
  • ; - match one uppercase alphabetic character [A-Z]
    , - match one lowercase alphabetic character [a-z]
    = - match one symbolic character [-_]
    ! - match one alphabetic or numeric character [a-zA-Z0-9]
With these wildcards you can reject players with auto-generated names like TommyTomato55, BillyHarrington121, or Player12 very easily:
  • Code: Select all

    try "It looks like you have an unofficial app. Please visit www.minetest.net for an ad-free playing experience!."
    
    fail any
    if $name is /;*;*##/
    if $name is /;*;*###/
    if $name is /Player#/
    if $name is /Player##/
    continue
    
    pass now
To censor names that begin or end with a symbol, as in "BobVila_"
  • Code: Select all

    fail any
    if $name is /*=/
    if $name is /=*/
    continue
To censor names with any symbols or numbers, as in "Bob-Vila"
  • Code: Select all

    fail any
    if $name is /*=*/
    if $name is /*#*/
    continue
To censor names with three or more uppercase letters in a row (uppercase spamming), as in "BOBVILA"
  • Code: Select all

    fail any
    if $name is /*;;;*/
    continue
To censor names with two or more symbols in a row (symbol spamming), as in "Bob____Vila"
  • Code: Select all

    fail any
    if $name is /*==*/
    continue
To censor names that don't begin with at least two letters, as in "1996_BobVila"
  • Code: Select all

    fail any
    unless $name is /&&*/
    continue
Please take note, the '~' wildcard has been deprecated and replaced with '&' to improve readability.

Pattern matching is now case-sensitive as well (this was necessary for the new wildcards to work properly). However, I am planning to introduce a flag to toggle this behavior in the near future. So, stay tuned.

In the meantime, the "is" comparison operator still supports case-insensitive comparisons when provided a string for the right-hand operand. The following condition would match for "ADMIN" and "admin" and "Admin" as it did previously.
I've also provided two more comparison operators for convenience when working with numbers. Both are self-explanatory.
  • gte - greater-than or equal
    lte - less-than or equal
Secondly, datasets should always be named with a ".txt" extension and must now be located within the "filters" subdirectory of your world. This separation should afford an added layer of security, not to mention better file system organization (particularly if you have several active or even dormant datasets). In an upcoming release, I will make the directory name configurable via minetest.conf.

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.3b (Authentication Handler) [auth_rx

by sorcerykid » Post

Version 2.4b Released

A new version of Auth Redux is ready for download. Here is a complete change log:
  • Moved Journal and AuthDatabase classes into library
  • Added rollback function to AuthDatabase class
  • Reworked journal audit to support rollback option
  • Better encapsulated database commit function
  • Allowed for STOPPED opcode during database update
  • Various changes to error and action messages
  • Moved command-line scripts to separate directory
  • Included script to rollback database from journal
  • Included script to extract debug log into journal
Several powerful database migration tools have been provided with this release. It is now possible to use your server's debug logs to extract player login activity and store the respective statistics directly into the Auth Redux database!

The original convert.awk script populated most fields with nil values. However, the new extract.awk script will analyze your debug.txt file and generate a transaction log that can be applied to the authentication database, giving you an accurate historic record of player login activity (dating as far back as your debug logs go, at least) -- including values for OldLogin, Lifetime, TotalSessions, and TotalFailures.
  • Image

    Image
I stress tested the script on the debug log from JT2 dating back to December 2016. The input file size was 11.0 GB, and it only took about 9 minutes and 30 seconds to complete the operation. Despite the myriad of server crashes and missing accounts that accumulated in my debug log before I developed this mod, it still churned out some impressive results!
  • Image
The parser is sophisticated enough to compensate for inconsistencies between the authentication database and the debug log. It will also detect server startup, shutdown, and abnormal termination events which are necessary for calculating session lengths. Unfortunately, client connections are not logged (in the vanilla minetest config), so TotalAttempts is the only statistic that is unavailable using this method.

For more detailed output from the script, you can change the debug level to 'verbose'. This will display both errors and warnings. It might be helpful to redirect output to a temporary file for this purpose. To see only errors, the default debug level of 'terse' should be sufficient.
  • Code: Select all

    > awk -f extract.awk -v debug=verbose /tmp/auth.txt /home/minetest/.minetest/debug.txt
Warnings occur if there are orphaned accounts in the 'auth.txt' file that do not appear in the corresponding debug log. This happens when the debug log is incomplete. Orphaned accounts will have no player login activity applied. Errors occur if the 'auth.txt' file is inconsistent with the debug log. This could be the result of a server crash or intentional deletion of accounts. Such accounts are deemed invalid and player login activity will be ignored. However, you should verify that you are using the correct 'auth.txt' file.
  • Image
    Image
Afterward, it is simply a matter of replaying the import journal against the newly converted database using the provided rollback.lua script. The import journal must reside in the same directory as the database to be updated.
  • Image
The rollback.lua script will automatically backup the import journal by renaming it "~auth.dbx". It will also prepare an empty journal within the same directory. You are now ready to start your Minetest server with a fully populated authentication database. Congratulations!

Here is the required sequence of commands to perform a conversion and extraction:
  • Code: Select all

    > cd auth_rx/tools
    > awk -f convert.awk -v mode=convert ~/.minetest/worlds/world/auth.txt
    > awk -f extract.awk -v debug=terse ~/.minetest/worlds/world/auth.txt ~/.minetest/debug.txt
    > lua rollback.lua ~/.minetest/worlds/world/auth.db
I also included instructions about this conversion and extraction process within the head of the extract.awk script.

For reference, the following table summarizes exactly what type of file operations each script performs:
  • Image
Notice that that the Auth Redux mod never directly manipulates files that were created by Minetest under any circumstances.

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.4b (Authentication Handler) [auth_rx

by sorcerykid » Post

Version 2.5b Released

A new version of Auth Redux is ready for download. Here is a complete change log:
  • Implemented function parsing algorithm for rulesets
  • Simplified operand matching logic in rulesets
  • Improved transcoding of literals in rulesets
  • Added some basic functions for use by rulesets
  • Fixed validation of dataset names in rulesets
I'm excited to announce that MARS now supports functional expressions within conditional statements. The syntax is consistent with Javascript, Perl, Python, and various other imperative programming languages:
  • func(arg1,arg2,...)
An argument can be either a variable, a literal, or a nested function, and it must evaluate to the correct datatype (string, number, etc.) as there is no implicit type conversion. For example, the add() function accepts two numbers, and returns the sum:
  • Code: Select all

    pass now
    if add(1,1) eq 2
    continue
I also provided an idiomatic "dataflow" operator for daisy-chaining functions together for sake of readability:
  • arg1->func1(arg2,...)->func2(arg3,...)
The dataflow operator has left-to-right associativity. So in the expression above, arg1 is evaluated and inserted as the first argument to func1. Then func1 is evaluated and the result is inserted as the first argument to func2.

For example, the following functional expressions are identical, although the first is entirely non-idiomatic:
  • Code: Select all

    mul(add(len("TEST"),2),neg(0.5))
    "TEST"->len()->add(2)->mul(0.5->neg())
A handful of builtin functions are available for the most common login filtering tasks:
  • sub(a,b)
    return a - b
  • add(a,b)
    return a + b
  • mul(a,b)
    return a * b
  • div(a,b)
    return a / b
  • neg(a)
    return the negative value of a
  • abs(a)
    return the absolute value of a
  • max(a,b)
    return the maximum value of a or b
  • min(a,b)
    return the minimum value of a or b
  • int(a)
    return the integer part of a
  • uc(a)
    return the uppercase of string a
  • lc(a)
    return the lowercase of string a
  • len(a)
    return the length of string a
  • trim(a,b)
    returns string a shortened by b characters at the end (or beginning, if negative)
  • crop(a,b)
    returns string a shortened to b characters from the beginning (or end, if negative)
By harnessing functional expressions, you can construct far more elaborate rulesets. Here are some ideas to get you started:
  • Code: Select all

    # reject players with names that are too short or too long
    
    fail any
    $name->len() gt 15
    $name->len() lt 5
    continue
    

    Code: Select all

    # reject new player registrations when server is nearing capacity
    
    fail all
    $is_new eq $true
    $cur_users gte $max_users->mul(0.8)
    continue

    Code: Select all

    # reject players with mostly-uppercase long names
    
    fail all
    $is_new eq $true
    len($name) gt 10
    $name is /*&&&&*/
    continue

    Code: Select all

    # reject players with all uppercase names
    
    fail any
    $name eq uc($name)
    continue
Using the lc( ) function it is also possible to do case-insensitive pattern matching. For example, the following comparison will match for "Admin" and "administrator" and "ADMIN":
Of course, the same results could be obtained using the crop( ) function:
I endeavored to keep the parser as lightweight as possible, with no external dependencies. Lua regex's don't natively support recursion. And despite hours of searching online, the only recommended solution I could find is a full-fledged parsing expression grammar library, like LPeg. I was convinced there had to be a less cumbersome means for a single use-case like this. So I ended up rolling my own algorithm which (much to my surprise) turned out to be just 25 lines of code. Not too shabby :)

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.4b (Authentication Handler) [auth_rx

by sorcerykid » Post

Version 2.6b Released

A new version of Auth Redux is ready for download. Here is a complete change log:
  • Iintroduced support for array literals in rulesets
  • Added array-related functions for use by rulesets
  • Localized references to transcoding functions
  • Registered chat command to control login filtering
  • Included support for disabling login filtering
  • Added reload function to AuthFilter class
  • Tweaked lexer to skip comments on ruleset loading
  • Added search function to AuthDatabase class
This release provides support for array literals, including several array-related functions. The syntax is similar to that of a function argument list. Elements can be either string literals, string variables, or nested functions that evaluate to a string.
  • (elem1,elem2,elem3,...)
The following three functions are available for working with arrays:
  • split(a, b)
    return an array derived from string a split by string b
  • size(a)
    return the number of elements in array a
  • elem(a,b)
    return element b from array a (or an empty string if nonexistent)
An array literal can be used anywhere that an array is expected, making it a suitable alternative to data streams, particularly when working with a small number of elements. For example, both of these expressions evaluate to "WORLD":
  • Code: Select all

    "Hello there world"->split(" ")->elem(3)->uc()
    
    ("Hello","there","world")->elem(3)->uc()
For added convenience, I defined two more preset variables:
  • $users_list - list of online players (array)
    $owner - name of the server operator (string)
There are many uses for both array literals and array-related functions. Here are just a few examples:
  • Code: Select all

    # reject players that do not have any privileges
    
    fail any
    if size($privs_list) eq 0
    continue

    Code: Select all

    # reject players that attempt to register with a reserved name (case insensitive)
    
    fail any
    when ("moderator","server","client","owner","player","system","operator","minetest") is $name
    continue

    Code: Select all

    # restrict administrator access by IP address (to safeguard against hacking)
    
    fail any
    if $name eq $owner
    until ("172.16.100.9","172.16.100.10","172.16.100.11") eq $addr
    continue

    Code: Select all

    # reject players with a 50% failed-login ratio unless the administrator is online
    
    fail all
    if $failures gte $sessions->div(2)
    unless $users_list eq $owner
    continue
A new "/filter" chat command allows you to enable or disable login filtering directly in game. It can also be used to reload a ruleset from disk. By default, login filtering is enabled at each server restart.
  • /filter [enable|disable|reload]
  • /filter
You can verify the status of login filtering by issuing the command without any parameters. This command requires the "server" privilege.

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.4b (Authentication Handler) [auth_rx

by sorcerykid » Post

Version 2.7b Released

A new version of Auth Redux is ready for download. Here is a complete change log:
  • Implemented time and date datatypes for rulesets
  • Updated code samples with latest feature-set
  • Added time-related functions for use by rulesets
  • Added time-related variables for use by rulesets
  • Minor formatting fixes to source code
  • Optimized comparison algorithm in ruleset parser
This release is a huge milestone in the evolution of MARS, as date and time support is now integrated directly into the language. In fact, MARS handles all time-related conversions behind the scenes, so even complex comparisons are a breeze. The following scenarios showcase just how easy it is to filter login requests based on relative or absolute times and dates:
  • Scenario 1:
    You want to deny users (except moderators, of course) from joining your server after 90 days of inactivity.

    Code: Select all

    try "Sorry, this account has been disabled by the administrator due to inactivity."
    
    fail all
    if age($newlogin) gt 90d
    until $privs_list eq "basic_privs"
    continue
    Scenario 2:
    After server restarts, you want to give VIP access to players that have spent at least 50 hours in-game.

    Code: Select all

    fail all
    if $uptime lt 15m
    unless $lifetime gte 50h
    continue
    Scenario 3:
    You want to block new players from joining your server on the weekends during known peak traffic hours.

    Code: Select all

    fail all
    if $is_new eq $true
    when ("Sat","Sun") eq day($clock)
    if time($clock) gte 09:00:00
    if time($clock) lt 18:00:00
    continue
    Scenario 4:
    You want to prevent malicious players from spam-logging your server (repeatedly joining and then leaving).

    Code: Select all

    try "You are doing that too much! Please wait awhile before logging in again."
    
    fail all
    if age($newlogin) lt 15s
    continue
The following specialized data types are available for working with times and dates:
  • Moment - An absolute time value (as in 12:30:00 01-Jan-2018)
  • Interval - A relative time value (as in 15 minutes)
  • Timespec - The time component of a moment (as in 12:30:00)
  • Datespec - The date component of a moment (as in 01-Jan-2018)
Interval literals are represented as a cardinal value with a suffix denoting the time scale
  • <value><scale>
Valid time scales are "y" for year, "w" for week, "d" for day, "h" for hour, "m" for minute, and "s" for second. Intervals are stored internally as seconds, so direct comparisons of literals are always possible, regardless of the time scale:
Moment literals are represented as an interval offset from the clock (minus prefix) or the epoch (plus prefix)
  • -<value><scale>
    +<value><scale>
In essence, a moment literal is syntactic sugar for the the before( ) and after( ) functions, as the following comparisons show:
  • Code: Select all

    -10d eq $clock->before(10d)
    +64d eq $epoch->after(64d)
The special variables $clock and $epoch always equate to the system clock and the UNIX epoch, the same values can be obtained from the moment literals -0s and +0s:
A UNIX timestamp can therefore be represented literally, as an offset interval from the epoch in seconds:
The special $y2k variable is a shorthand for the moment representing January 1, 2000. So the year 2018 could be expressed as
Alternatively, the at( ) function accepts a full string representation of a date and time in ISO 8601 format for added convenience:
A timespec is represented as a cardinal value in the customary hours, minutes, and seconds notation. The alternate notation, without seconds, is accepted as well. Timespecs are only used for purposes of comparison and do not store an actual timestamp.
  • <hours>:<minutes>:<seconds>
    <hours>:<minutes>
A datespec is represented as an ordinal value in the day, month, year notation.
  • <day>-<month>-<year>
The day can be one or two digits, whereas the month must be two digits and the year must be four digits. Datespecs, like their counterparts, are used only for purposes of comparison.

To simplify the conversion between these four datatypes, several helper functions are provided:
  • date(a)
    converts moment a to a datespec
  • time(a)
    converts moment a to a timespec
  • age(a)
    calculates the interval between the system clock and moment a
  • day(a)
    converts moment a to a three-letter string representing the weekday (locale dependent)
  • before(a,b)
    subtracts interval b from moment a
  • after(a,b)
    adds interval b to moment a
  • at(a)
    converts string a in ISO 8601 combined format to a moment (locale dependent)
You can use the date() and time() functions to compare timespecs or datespecs against moments. Say, you want to determine if yesterday was August 1, 2018.
The before() and after() functions allow for basic arithmetic of moments and intervals. Maybe you want to determine whether the server was started in the morning.
  • Code: Select all

    if time($clock->before($uptime)) lt 12:00
In summary, here are all the time-related variables at your disposal:
  • oldlogin - timestamp of the original login (moment)
  • newlogin - timestamp of the latest login (moment)
  • lifetime - length of time the player has spent in-game (interval)
  • epoch - timestamp of the UNIX epoch (moment)
  • clock - timestamp of the system clock (moment)
  • uptime - length of time the server has been online (interval)
Important Note: The values of $clock, $y2k, and $epoch are based on your local time zone, not UTC. When in doubt, MARS uses the same time and date that is recorded in the Minetest debug log.

To wrap up these "time-sensitive" release notes, here is a brief public service announcement from the Mad Hatter, reminding you of the importance of proper time keeping!

Image

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.4b (Authentication Handler) [auth_rx

by sorcerykid » Post

Version 2.8b Released

A new version of Auth Redux is ready for download. Here is a complete change log:
  • Introduced support for numeric patterns in rulesets
  • Created polymorphic pattern-matching classes
  • Excluded time-zone offsets in time/date comparisons
  • Standardized timestamps for use in filter functions
  • Removed daylight saving time from preset variables
  • Updated comparison algorithm in ruleset parser
  • Passed preset variables array to filter functions
I've greatly expanded the date and time capabilities of MARS, by rolling out multi-field numeric pattern matching. If you are familiar with the syntax for crontabs, then you will feel very much at home with this release :)

Pattern Matching Modes

All pattern matching modes adhere to a consistent syntax, with a glob between two forward slashes followed by a single-letter indicating the parser to use:
  • /<string>/s

    /<hours>:<minutes>:<seconds>/t

    /<day>-<month>-<year>/d
For string comparisons, the "s" may be omitted (it is the default parser, to ensure backward compatibility). For time and date comparisons, the "t" and "d" parsers must be explicitly specified.

An all-too-common use case would be to set opening-hours for a server:

Code: Select all

# only accept connections between 8:00 and 20:59 daily (except administrators)

fail all
if $clock is /8^20:?:?/t
until $privs_list eq "basic_privs"
continue
Numeric pattern matching is particularly sophisticated, allowing for a variety of range checks against individual fields:
  • a
    exact match

    a^b
    match between a and b inclusive

    a>
    match greater than or equal to a

    a<
    match less than or equal to a

    ?
    indefinite match
Perhaps you want confirm that a player's initial login was between July and December of 2017.
  • Old method:

    Code: Select all

    if date($oldlogin) gte 01-06-2017
    if date($oldlogin) lte 31-12-2017
    New Method:

    Code: Select all

    if $oldlogin is /?-06^12-2017/d
Then again, you might want to know whether the current system time is the afternoon or evening:
  • Old method:

    Code: Select all

    if time($clock) gte 12:00:00
    New method:

    Code: Select all

    if $clock is /12>:?/t
Notice how the time and date parsers match a moment directly. Hence the time( ) and date( ) wrapper functions are unnecessary. This is not just for sake of convenience, but it was imperative to avoid multiple daylight-saving time computations (as both operands would need to be synchronized, which is overkill).

Speaking of daylight saving time, MARS does all of the grunt work for you behind the scenes. In fact, I've verified over 40 different time and date related equations to squash any potential bugs.
  • Code: Select all

    # Success
    if time(at("2018-07-20T00:00:00Z")->after(1d)->after(2h)) eq time(at("2018-07-21T02:00:00Z"))
    
    # Success
    if time(at("2018-07-20T00:00:00Z")->before(1h)) eq 23:00:00
    
    # Success
    if date(at("2018-07-20T00:00:00Z")) eq date(+1532131200s->before(24h))
    
    # Success
    if date(at("2018-07-20T00:00:00Z")) eq 20-07-2018
    
    # Success
    if "2018-07-20T0:30:00Z"->at()->after(2h) is /2:?:?/t
    
    # Success
    if day(at("2018-07-20T0:30:00Z")->before(1d)) eq "Thu"
    
    # Success
    if $epoch->after(10y) is /0:?:?/t
To my knowledge there is only one scenario where you will encounter problems, and that is with time and date arithmetic. The before( ) and after( ) functions, with large inteverals (like 150 days) could produce results that are either one hour in advance or behind what you might expect, due to crossover from daylight saving to standard time.

This is arguably the expected behavior since these two functions are performing arithmetic on timestamps, not on a wall clock. And in the real world, if you engage a stopwatch on March 10, and wait exactly 24 hours, the clock will read a one hour difference from the stopwatch. Hence, I don't really consider this a "bug", but rather a helpful feature ;D

Benchmarking Results

With this release I also did some benchmarking of the MARS interpreter, to assess its performance during Minetest prejoinplayer callbacks.

Even with a ruleset file in excess of 110 widely varying conditions (both simple and complex) including a blacklist of 100 entries, the parser and lexer consistently clocked between 0.004s and 0.007s! In fact, the entire MARS interpreter has an on-disk footprint of a mere 20 kilobytes. So as a full-fledged ruleset-based programming language (coded entirely in Lua), it is remarkably lightweight and efficient. Although, I am always working on further optimizations, of course :)

Image

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.8 (Authentication Handler) [auth_rx]

by sorcerykid » Post

Version 2.9b Released

A new version of Auth Redux is ready for download. Here is a complete change log:
  • Implemented address datatype for rulesets
  • Added more helper functions for use by rulesets
  • Fixed missing syntax checks in ruleset parser
  • Developed and integrated AuthWatchdog class
  • Added meta-variables for stateful login filtering
I'm very excited to announce that Auth Redux now supports stateful login filtering by IP address, making it perhaps the most sophisticated framework for rule-based pre-authorization of any online gaming platform. With an exhaustive suite of preset variables and builtin functions, server administrators now have both flexible and fine-grained access control.

Stateful Login Filtering

Consider, the frequent scenario of blocking users after multiple failed logins. In a purely Lua-driven approach, this type of mechanism would not only entail hours of coding and testing. But it would become a maintenance (and security) headache. In MARS, however, it's just a few lines of code, virtually no effort to extend or adapt to any number of use-cases:
  • Code: Select all

    # block players for 45 seconds after 2 or more failed logins from the same IP address
    
    fail all
    if $ip_attempts gt 0
    if $ip_failures gte 2
    if age($ip_newcheck) lt 45s
    continue
    
That's all there is to it. Just start your Minetest server, and you're ready go, while Auth Redux handles the rest!

The new Watchdog class monitors all login activity on the basis of IP address, tracking the time and state of each login request. This makes it very easy to intercept would-be hackers, preventing them from ever reaching the password verification phase.

Six meta-variables are provided by the Watchdog class with state-based information specific to the IP address:
  • $ip_prelogin - the timestamp of the last attempted login
  • $ip_oldcheck - the timestamp of the first failed login
  • $ip_newcheck - the timestamp of the last failed login
  • $ip_failures - the current number of failed logins
  • $ip_attempts - the current number of attempted logins
  • $ip_names_list - list of assumed usernames in chronological order
All meta-variables are persistent until the user successfully logs into the sever, at which point they are reset. If the server is restarted, they will be reset as well (since they are non-essential).

Keep in mind, an attempted login is recorded by the Watchdog class directly after the login filtering phase, yet prior to the password verification phase. Hence, if a user at 127.0.0.1 connects to your server for the first time, the $ip_oldcheck, $ip_newcheck, and $ip_prelogin variables will be undefined. Therefore, rules involving the $ip_prelogin variable must first validate $ip_attempts (it will be 0 if this is a newly attempted login). Furthermore, if a login request is rejected by a ruleset, then the password verification phase will aborted. Hence, you must also validate $ip_failures prior to using the $ip_oldcheck and $ip_newcheck variables in your rules.

To aid in analyzing the $ip_names_list variable, I added two more helper functions for working with arrays:
  • clip(a,b)
    extract b number of elements from the beginning of array a (or the end if negative)
  • count(a,b)
    count the number of occurrences of string b in array a
It should be noted that the count( ) function is not a replacement for the where/until statement, since it iterates over the entire array.

Tomorrow I will writeup a step-by-step tutorial on how to get the most out of these new-meta variables. There are numerous different applications, so it will take a separate post to fully document everything. Stay tuned!

New Address Datatype

On top of the aforementioned changes, I added a new address datatype. I figured it was worth the extra bit of code, since IP addresses (as elucidate above) are arguably one of the most important aspects of login filtering.

This means that addresses also include a pattern matching parser. Similar to the time and date parsers described above, all of the same wildcards and range checks are supported. You can filter requests by specific ranges of IPs or entire subnets very easily:
  • Code: Select all

    fail any
    if $addr is /172.16.?.?/a
    if $addr is /192.168.12<.?/a
    if $addr is /203.0.113^114.?/a
    continue
Of course, address literals are supported as well:
  • Code: Select all

    pass any
    if $addr eq 192.88.99.1
    if $addr eq 192.88.99.2
    continue
Existing rulesets will need to be updated accordingly. Or alternatively, you can use the new ip( ) wrapper function to convert $addr variables into a string for comparisons against arrays.
  • Code: Select all

    if ip($addr) eq "192.88.99.1"
    
    when ("192.88.99.1","192.88.99.2") eq ip($addr)
In a forthcoming version, I will add support for arrays of non-string datatypes. So this is just an interim workaround.

Thanks for reading! I really hope you enjoy these new features :)

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.9 (Authentication Handler) [auth_rx]

by sorcerykid » Post

Tutorial on Limiting Login Attempts

Protecting your Minetest server against brute force attacks, as with any online service, necessitates limiting the number and frequency of attempted logins. By automatically blocking potentially malicious users during pre-authorization, there is less likelihood that any account on your server will be compromised. This is particularly crucial for trusted users, such as moderators and administrators.

The Watchdog feature of Auth Redux provides a complete set of meta variables (as explained above) for detecting suspicious login activity prior to the password-verification stage. This tutorial will offer a step by step guide for improving the security of your Minetest server by pro-actively limiting login attempts.

We'll begin by building a very basic rule that blocks any users that fail to login 3 times from the same IP address:
  • Step 1:

    Code: Select all

    try "Please wait awhile before trying to login again."
    fail all
    if $ip_failures gte 3
    continue
    Tip: $ip_failures is the total number of failed logins by the connecting IP address
We're already off to a great start! But we still have do some fine tuning. Without an expiration, even innocent users will be barred indefinitely. That's not too helpful. So let's impose a 5 minute lockout period after 3 failed logins.
  • Step 2:

    Code: Select all

    try "Please wait awhile before trying to login again."
    fail all
    if $ip_failures gte 3
    if age($ip_newcheck) lt 5m
    continue
    Tip: age($ip_newcheck) is the time elapsed since the last failed login by the connecting IP address
This is a remarkable improvement. But, what if this user has multiple accounts? It would be more sensible to restrict access to only the first account, while giving the user an opportunity to login to another account instead.
  • Step 3:

    Code: Select all

    try "Please wait awhile before trying to login again."
    fail all
    if $ip_failures gte 3
    if age($ip_newcheck) lt 5m
    when $ip_names_list eq $name
    continue
    
    Tip: $ip_names_list is a chronological list of all assumed usernames by the connecting IP address
At this point, if the user fails to login to the second account, then both accounts will be restricted for 5 minutes. Since this is less likely to be a brute force attack, we could be somewhat lenient by checking the most recent username.
  • Step 4:

    Code: Select all

    try "Please wait awhile before trying to login again."
    fail all
    if $ip_failures gte 3
    if age($ip_newcheck) lt 5m
    if $ip_names_list->elem(-1) eq $name
    continue
    
    Tip: $ip_names_list->elem(-1) is the last assumed username by the connecting IP address
Now the user can retry any failed login, by switching between accounts. Nevertheless, a would-be attacker will likely exploit this loophole. To mitigate the potential for abuse, let's instead place a hard limit of 3 attempts per-username:
  • Step 5:

    Code: Select all

    try "Please wait awhile before trying to login again."
    fail all
    if $ip_failures gte 3
    if age($ip_newcheck) lt 5m
    if $ip_names_list->count($name) gte 3
    continue
    Tip: $ip_names_list->count($name) is the number of times this username was assumed by the IP address
We should realistically lookup only the last 20 usernames, rather than iterating over the entire list. Although Lua can traverse arrays of thousands of items very efficiently, it's still a worthwhile safeguard against DoS-style attacks:
  • Step 6:

    Code: Select all

    try "Please wait awhile before trying to login again."
    fail all
    if $ip_failures gte 3
    if age($ip_newcheck) lt 5m
    if $ip_names_list->clip(-20)->count($name) gte 3
    continue
    
Now the user can log into another account during the 5 minute window, with no additional penalty. If the user fails again, the lockout period will reset. To make this a full-fledged ruleset, we just have to add a "pass now" rule to the end.
  • Step 7:

    Code: Select all

    try "Please wait awhile before trying to login again."
    fail all
    if $ip_failures gte 3
    if age($ip_newcheck) lt 5m
    if $ip_names_list->clip(-20)->count($name) gte 3
    continue
    
    pass now
Everything looks great! We've achieved a decent compromise between security and convenience. As a final improvement, let's weed out nuisance users that create new accounts whenever they forget their password. A lockout period of 10 minutes should suffice.
  • Step 8:

    Code: Select all

    try "You cannot create a new account right now."
    fail all
    if $is_new eq $true
    if $ip_failures gt 0
    if age($ip_oldcheck) lt 10m
    continue
    
    try "Please wait awhile before trying to login again."
    fail all
    if $ip_failures gte 3
    if age($ip_newcheck) lt 5m
    if $ip_names_list->clip(-20)->count($name) gte 3
    continue
    
    pass now
And there we have it! In just eight steps, we've helped to protect our server against the threat of most brute-force attacks. Best of all, this ruleset is simple enough to customize as our server grows and our needs change.

I hope this tutorial was helpful! If you have any questions, please feel free to ask :)

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.9 (Authentication Handler) [auth_rx]

by sorcerykid » Post

Version 2.10b Released

A new version of Auth Redux is ready for download. Here is a complete change log:
  • Major code reorganization via multiple libraries
  • Removed extra variables from AuthFilter class
  • Developed interactive debugger for testing rulesets
  • Added optional debugger hooks in AuthFilter class
  • Allowed for overriding preset variables by debugger
  • Included line-number in results of login filter
  • Added missing preset variable needed by rulesets
I was hard at work this weekend developing an integrated debugger. Now you can test your ruleset definitions interactively, without the hassle of launching a new server instance for each and every trial. The debugger attempts to simulate the entire login filtering process within a sandboxed environment.

Image

Important Note: The debugger requires version 2.3 or higher of the ActiveFormspecs Mod be installed.

By entering the "/fdebug" command into chat (requires the "server privilege"), you will be presented with a debugging console. The workspace consists of the following elements:
  • A. The "Show Client Output" option toggles whether to display the client output panel. The "Show Debug Prompt" option toggles whether to insert debug status prompts into the source code.

    B. This textarea contains the ruleset definition to be examined. Although Minetest supports editing of text, it is strongly recommended to copy and paste your source code into a full-fledged text editor.

    C. The client output panel renders error messages as they would appear within the client. The status panel typically indicates whether the ruleset passed or failed, as well as other debugging conditions.

    D. The "Save" button will export the current ruleset definition, overwriting "greenlist.mt" in your world directory. The "Load" button will import an existing ruleset definition from "greenlist.mt" for debugging.

    E. The "Process" button will process the ruleset definition according to the selected login filtering criteria: Normal, New Account, or Wrong Password (thereby changing the relevant preset variables).

    F. The preset variables are listed here with their corresponding values. These values will never change except during the login filtering process, or unless explicitly set in the panel below.

    G. The name and type of the selected variable is indicated here. The value can be edited in the text field, and set with the "Set" button. The arrow buttons allow for re-ordering any variable within the list.
Some variables, like $clock and $uptime, have an "Auto Update" option to toggle whether the values should be derived from the system state. For a fully sandboxed environment, you can disable this option.

The special $__debug variable gives you direct access to the MARS expression interpreter. You can enter any valid expression, and the resulting value and type will be displayed in the panel above (all variable types, except patterns, are supported). This is particularly helpful for monitoring the values of certain variables. To calculate the size of the $ip_names_list, for example, you would enter
Whenever a ruleset passes or fails, or if a syntax error is encountered, a debug status prompt will be inserted into the source code below the line in question:
  • Code: Select all

    if $name->len() eq "administrator"
    # ====== ^ Line 12: Mismatched operands in ruleset^ ======
These breakpoints will be removed automatically when the ruleset definition is saved, so there is no need to edit them out.

I tested this debugger in three different builds of Minetest under both Linux and Windows. Overall, the performance is on par with the runtime environment. This is to be expected since it hooks directly into the AuthFilter API, injecting different preset variables.

Since "/fdebug" is a fully-encapsulated chat command, the debugger itself doesn't incur any additional overhead if you choose not to use it, other than being a registered callback of course. It just waits patiently at the sidelines, hoping that you drop by for a visit sometime ;)

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.11 (Authentication Handler) [auth_rx

by sorcerykid » Post

Version 2.11 Released

The transaction log of Auth Redux has always served a dual purpose. Besides protecting against data loss and corruption, it also permits efficient reporting and analysis. With this release, I've included a command-line analytics script for just this purpose!

You can review wide variety of server and player metrics for any given 24-hour period:
  • Total Players
    total number of unique players that joined per day
  • Total New Players
    total number of new players that joined per day
  • Total Player Sessions
    total number of player sessions per day
  • Total Login Failures
    total number of failed logins per day
  • Total Login Attempts
    total number of attempted logins per day
  • Overall Server Uptime
    aggregate uptime percentage of the server per day
  • Maximum Connected Clients
    maximum number of connected clients per day
  • Minimum Connected Clients
    minimum number of connected clients per day
  • Maximum Player Lifetime
    maximum session duration amongst all players per day
  • Average Player Lifetime
    average session duration amongst all players per day
Most of these statistics are also broken up into an hour-by-hour format. The following statistics are player-specific:
  • Total Sessions
    total number of sessions of each player
  • Total Lifetime
    aggregate session duration of each player
Generating reports of player login activity is both quick and easy. It can even be accomplished while the server is still online. From the shell, change to the tools subdirectory and issue the following command:

Code: Select all

cd auth_rx/tools
awk -f report.awk -v days=1 -v type=txt ~/.minetest/worlds/new_survival/auth.dbx
Given a "days" parameter of 1 and a type parameter of "txt" ,this will analyze the journal file in the "new_survival" world, and generate a plain-text report of player login activity for yesterday. Changing the "days" parameter to 0 will obtain data from today. So on and so forth.

Here is an example of the results for the JT2 server on April 7, 2018.

Image
Image
Image
Image

You can optionally redirect output to a file or even pipe to another command. This would be useful for receiving automated daily reports by email. Here's a quick tutorial on how to set this up as a cron job:

https://www.nixtutor.com/linux/sending- ... ough-cron/

By changing the "type" parameter to "js, the results will be formatted as a JSON for embedding into a dynamic Website. Here is a very simple DHTML page with an AJAX request. Of course you are free to custom-tailor the output entirely to your needs.

Image

The structure of the JSON is as follows (with some sample values included):

Code: Select all

{
	"global_stats": {
		"total_players": 2,
		"total_players_new": 0,
		"total_sessions": 2,
		"total_failures": 0,
		"total_attempts": 2,
		"server_uptime": 120,
		"max_clients": 1,
		"min_clients": 0,
		"max_lifetime": 60,
		"avg_lifetime:" 38
	},
	"player_stats": {
		"test_user1": {
			"sessions": 1,
			"lifetime": 60
		}
		"test_user2": {
			"sessions": 1,
			"lifetime": 15
		}
	},
	hourly_stats: [
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 2, failures: 0, attempts: 1, players: 2, clients_min: 0, clients_max: 1 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 },
		{ sessions: 0, failures: 0, attempts: 0, players: 0, clients_min: 0, clients_max: 0 }
	]
}
Finally Out of Beta! As of this release, I've decided to move Auth Redux out of beta since there are no known issues that impact security or stability to date. Wh00t! New features are still planned of course, but no compatibility-breaking changes insofar as the underlying database and journal architecture are concerned.

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.11 (Authentication Handler) [auth_rx

by sorcerykid » Post

Version 2.12 Released

I just completed several breaking-changes to the MARS interpreter which I feel are long overdue. Some of these ideas were part of my original specification for the language, but they weren't implemented for a variety of reasons (in great part because I was long past the deadline for release, and I had to settle on a convention). However, I feel this syntax is more consistent, straightforward, and concise.

Important! These updates may impact compatibility with existing ruleset definitions, so please read carefully below. If in doubt, I would recommend testing your rules using the /fdebug chat command described above prior to deployment.

Single-Condition Rules

The original "when" and "until" statements have been deprecated. However, they now serve as abbreviated rules instead. If you have many rules with just one condition, this should afford a significant reduction of code.
  • Current Syntax (Still Supported):

    Code: Select all

    pass any
    if $name eq "admin"
    continue
    New Syntax:

    Code: Select all

    when $name eq "admin" pass
All of the usual comparison operators are available. However, there is no need for a "continue" statement or a boolean specifier.

Membership Comparisons

The new "in" operator takes an array as its right-hand operand and a string for the left. Like the other comparison operators, it can appear within an "if" or "unless" statement as well as the new "while" or "until" statements.
  • Deprecated Syntax:

    Code: Select all

    pass any
    when @whitelist.txt eq $name
    when $privs_list eq 'basic_privs'
    continue
    New Syntax:

    Code: Select all

    pass any
    if $name in @whitelist.txt
    if 'basic_privs' in $privs_list
    continue
Notice how the order of operands is inverted. Otherwise, membership comparisons are equivalent to their former counterparts.

Pattern Matching Against Arrays

Last but not least, the new "has" operator allows for pattern matching of strings against arrays. It can also be used for case-insensitive membership comparisons of strings, analogous to the "is" comparison operator.
  • Deprecated Syntax:

    Code: Select all

    fail all
    if $is_new eq $true
    when ('admin','sysadmin','server','operator') is $name
    continue
    New Syntax:

    Code: Select all

    fail all
    if $is_new eq $true
    if ('admin','sysadmin','server','operator') has $name
    continue
While these changes are a nominal inconvenience, they've allowed me to consolidate a fair amount of redundant code so that many other backend-optimizations are now possible. In a long term, I feel it's a win-win. Thanks again for bearing with me!

User avatar
sorcerykid
Member
Posts: 1841
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: [Mod] Auth Redux v2.12 (Authentication Handler) [auth_rx

by sorcerykid » Post

Auth Redux vs SQLite3

With all of the recent changes to Auth Redux, I decided to benchmark its performance, in order to weed out any potential bottlenecks. So I put it to the test against two different SQLite3 implementations using a database of over 522,000 records from the JT2 server

Much to my surprise, Auth Redux doesn't suffer at all when it comes to raw speed. It's insanely fast compared to both sauth and the new Minetest 5.0 builtin. For every test, it vastly out-performed these other authentication handlers. I'm talking orders of magnitude in the hundreds and even thousands, as shown by these baseline measurements:
  • Image
Even the database file size of Auth Redux is just 2/3rds that of auth.sqlite used by Minetest 5.0. And yet the schema is far more extensive (including total time spent playing, initial login timestamp, total player sessions, etc.) When it comes to rsyncing files in the hundreds of megabytes over SSH, this can make a huge difference.

Image

Perhaps, the biggest area of concern for most admins is the execution time during player joins. If a high-traffic server averages around 30 players joining every minute at peak hour, sauth would consume over 20 seconds of CPU cycles (that is 1/3 of server time spent processing authorization). Meanwhile, auth_rx clocks in at just 3.8 milliseconds after 30 player joins.

For each test below, I performed five independent trials and averaged the results to get the most accurate figure possible.

Image

Image

Image

As these benchmarks reveal, there is no discernible benefit to using an SQLite3 backend for authentication, whether in terms of reliability or performance. If anything, SQLite3 databases are a hindrance when it comes to random access of large volumes of data, and more than likely the culprit for why some high-traffic servers lately are experiencing notable lag spikes.

User avatar
TenPlus1
Member
Posts: 3715
Joined: Mon Jul 29, 2013 13:38
In-game: TenPlus1
Contact:

Re: [Mod] Auth Redux v2.12 (Authentication Handler) [auth_rx

by TenPlus1 » Post

This is something that the Minetest devs have to take into consideration, especially if it improves speeds to this degree.

Astrobe
Member
Posts: 570
Joined: Sun Apr 01, 2018 10:46

Re: [Mod] Auth Redux v2.12 (Authentication Handler) [auth_rx

by Astrobe » Post

A Lua script that is one or two order of magnitudes faster than an Sqlite3 database?

Post Reply

Who is online

Users browsing this forum: No registered users and 25 guests