Archive by month

I wanted to see what my mostly stock Drupal server was capable of.  For this first test I am starting with a simple homepage load. I used the ab tool (Apache Bench).  It couldn't be simpler!  But it only downloads the HTML, not the rest of the assets.  Here is one page load:

user@hostname:~/$ ab -n 1 -c 1 https://aaron.axvigs.com/
This is ApacheBench, Version 2.3 <$Revision: 1923142 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking aaron.axvigs.com (be patient).....done

Server Software:        nginx/1.22.1
Server Hostname:        aaron.axvigs.com
Server Port:            443
SSL/TLS Protocol:       TLSv1.3,TLS_AES_256_GCM_SHA384,256,256
Server Temp Key:        X25519 253 bits
TLS Server Name:        aaron.axvigs.com

Document Path:          /
Document Length:        44878 bytes

Concurrency Level:      1
Time taken for tests:   0.092 seconds
Complete requests:      1
Failed requests:        0
Total transferred:      45304 bytes
HTML transferred:       44878 bytes
Requests per second:    10.87 [#/sec] (mean)
Time per request:       92.023 [ms] (mean)
Time per request:       92.023 [ms] (mean, across all concurrent requests)
Transfer rate:          480.77 [Kbytes/sec] received

Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:        3    3   0.0      3       3
Processing:    89   89   0.0     89      89
Waiting:       86   86   0.0     86      86
Total:         92   92   0.0     92      92
 

45KB of data transfered.  This doesn't really match what Firefox shows in F12 (18KB transfered, 62KB size), so I am confused.  Anyways, now let's run it 5000 times.

user@hostname:~/$ ab -n 5000 -c 50 https://aaron.axvigs.com/
This is ApacheBench, Version 2.3 <$Revision: 1923142 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking aaron.axvigs.com (be patient)
Completed 500 requests
Completed 1000 requests
<snip>
Completed 5000 requests
Finished 5000 requests

Server Software:        nginx/1.22.1
Server Hostname:        aaron.axvigs.com
Server Port:            443
SSL/TLS Protocol:       TLSv1.3,TLS_AES_256_GCM_SHA384,256,256
Server Temp Key:        X25519 253 bits
TLS Server Name:        aaron.axvigs.com

Document Path:          /
Document Length:        44878 bytes

Concurrency Level:      50
Time taken for tests:   189.289 seconds
Complete requests:      5000
Failed requests:        3
  (Connect: 0, Receive: 0, Length: 3, Exceptions: 0)
Non-2xx responses:      3
Total transferred:      226385060 bytes
HTML transferred:       224255867 bytes
Requests per second:    26.41 [#/sec] (mean)
Time per request:       1892.886 [ms] (mean)
Time per request:       37.858 [ms] (mean, across all concurrent requests)
Transfer rate:          1167.95 [Kbytes/sec] received

Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:        2  270 1141.2      3   35655
Processing:    85 1491 2484.0   1450   60003
Waiting:       83 1488 2484.0   1448   60003
Total:         88 1761 2747.3   1473   63081

Percentage of the requests served within a certain time (ms)
 50%   1473
 66%   1519
 75%   1563
 80%   1614
 90%   2281
 95%   3852
 98%   7524
 99%  15358
100%  63081 (longest request)
 

Three VMs are involved in servicing the requests to my desktop PC.

• Application server
  • nginx/fast-cgi/php-fpm
  • Physical host was Dell Optiplex SFF with Intel i7-8700 (6 cores 3.2GHz).
  • 2 CPUs, runs at 50% during the test.  Possible single-threaded issue here.
  • 1GiB RAM, 80% used.  No change when testing starts.
• Database server
  • Physical host was QOTOM Everything 1U server with Intel Atom C3758R (8 cores 2.4GHz).
  • 3 CPUs, runs at 80-100% during the testing and 5-10% normally.
  • 16GiB RAM.  Disk activity doesn't change so presumably the relevant parts of the database are entirely in RAM.
  • Network traffic jumps up to 25MB/s (200Mbps) according to Proxmox.
• Reverse proxy server
  • nginx
  • Physical host was Supermicro E300 with Intel Atom C3558 (4 cores @2.2GHz).
  • 2 CPUs, runs at 15 percent during testing and 2% normally.
  • 512MiB RAM, no change in usage for testing.
  • 1.5MB/s of network bandwidth.

Yeah the physical hosts are nothing amazing.  They aren't heavily loaded by other VMs at least.  5-10% average CPU usage.

It appears to be limited by the database server.  This normally sits at 5-10% CPU usage, but runs at 100% during the test.  28 somethings per second seems alright.

Testing with some software that actually loads all the page resources is on my to-do list.

Since 2019 I have been running TT-RSS.  I started with it on a small Digital Ocean node as I was living on a sailboat at the time.  Once I had a house and ISP in 2021 it became one of ~10 VMs that I ran on an old laptop via Debian KVM/QEMU, and a couple years later when I started ramping up my Proxmox cluster and things moved over there.

Also of note: I run a separate PostgreSQL VM with a bunch of databases on it and the TT-RSS database is one of those.  And I just run the PHP code straight in the VM, no extra Docker layer.

TT-RSS turbulence

So a few months ago the notorious TT-RSS developer called it quits.  I do believe that it will continue to be maintained, but I was pretty sure there are better options.  The TT-RSS code is spaghetti.  I am somewhat accepting of this because it is similar to the kind of code I tend to write.  I am just an OK programmer.  So the TT-RSS code is like the kind where you can easily find the lines that do real work, because there are 5,000 lines of it in one file.  I have observed that "high-quality" code gets so abstracted out that it nearly all looks like boilerplate.  Overall I don't mind the quality of the codebase but it is a signal.

Another issue was the Android app.  I have been changing phone about once a year and there is a TT-RSS app somewhere (F-Droid?) that seems legit but doesn't work.  This always bogs me down for a bit.  Now the app is just discontinued?  Probably I could figure it out but it shouldn't be that hard.

Where to?

I didn't do a super thorough search. I wanted something self-hosted, database-backed, very OSS, etc.  I found FreshRSS and the demo site seemed OK.  There are lots of knobs to twiddle.  There are extensions.  The GitHub page looks lively.  Someone wrote a script to port all feeds and articles from TT-RSS to FreshRSS.  Might as well!

(I like to think of my feed reader as an archive of sorts.  Maybe 20 of the feeds I have followed over the years have now gone offline entirely.  It's kind of a hoarding thing, and kind of a nostalgia thing.  I like it.  So I wanted to keep all those articles.)

Migration script

Back to that script.  Here it is.  It worked pretty well and I wrote up my experience in an issue (an abuse of the issue feature I'm sure, sorry!).

I used this script to migrate 125,000 articles from 436 feeds. A batch size of 60000 yielded 3 files of 456MB, 531MB, and 62MB. This ran in under 1 minute and pretty much maxed out the 4GB of RAM on my TT-RSS VM.

So with these file sizes I had to bump all the web server file size limits to 1000MB, PHP memory limit to 1200MB, and various timeout values to 300 seconds. This was fine for the first file, but the second one maxed out that memory limit so I reran with a 3000MB limit. Also by this point I was worried about the 300 seconds being enough so I bumped that to 3000 seconds. And that file ran fine then too. It shouldn't be hard for most people to get better import performance as my database VM storage is slow.

I imported into FreshRSS version 1.28.0. I see that your first patch has been merged, but the second one about the categories still has not. I opted not to figure out how to get the category import to work and will manually categorize them. Yes, all 436 feeds. First of all I will take the opportunity to categorize better than before. Also, I had nested categories in TT-RSS and I'm not sure how that was going to translate to FreshRSS anyways.

Thank you for the script!

Overall it went pretty smooth and I have been using it for about a week.  I am happy with it!  I have most of my feeds categorized and the dead ones "muted" (should be called "disabled" or something).

Duplicate articles

I did notice an issue with duplicate articles affecting a lot but not all feeds.  Say a feed on a website gives the 10 latest articles.  TT-RSS had pulled all 10 of those and had them in the database, along with say 90 past articles.  So those 100 articles were in the TT-RSS database and were copied to FreshRSS.  Then FreshRSS pulled the feed from the website and didn't recognize that those 10 articles already existed in the database, and stuffed them in again.  So now there are 10 articles listed twice.

This was a little upsetting but after two days I don't really notice it anymore.  I'm guessing that TT-RSS did something dumb with article GUIDs but could be wrong.

Good UI

The FreshRSS UI is pretty good, definitely better than TT-RSS.  So I have already set up two features that I just hadn't bothered with in TT-RSS.  I have been disabling old feeds so that they don't show up as errored out.  And I have set up a few feeds to automatically download full articles when they aren't included in the feeds.  This is actually super simple, no extension needed.

Other notes

I miss nested feed categories but don't really need them.

I would love to save article images to my personal S3-compatible archives but there doesn't seem to be an extension capable of doing this.

I haven't set up a mobile app yet.  I have heard the website is good enough on mobile so will try that first.

I am not using any extensions yet.

I moved my Digikam database from the local SQLite file to a networked MariaDB server.  This was mostly fine but Digikam (8.8.0 version) was performing terribly after any face tagging operation.

I would get frequent Force Quit or Wait? questions from the OS, and the UI would hang for 5-10 seconds.  This was always matched with a face scan job kicking off and showing up in the jobs queue.  Everything generally worked but was incredibly tedious.  ~20,000 images in my collection, BTW.

A few times I went through the DB's config and optimized some things, which helped a little.  Typical tuning things like increasing memory and threads, and bumping up max_allowed_packet as recommended.  Still the issue persisted.

Eventually I learned that there is a Enable background face recognition scan setting.  It seems obvious in retrospect, but this completely fixed the issue.  I will just need to periodically rescan for faces now to apply what training has been done since the last run.