while troylu.com is under renovations, this blog will live here temporarily
nearsay - mishaps and terrible design decisions
As basically my first full stack project, lots of mistakes were made. I've recorded some of them here.
Or at least, the more interesting ones that aren't just "whoops, I committed and pushed my .env
file with critical secrets haha"
(which I maybe did. maybe more than once)
someone else already solved that problem, and way better than you can
When I first started thinking about how to query for all the posts visible within the user's rectangular screen, I had the idea of splitting the region into tiles to make the search quicker.
This led to writing nearby-points, an npm package that I thought would be a great solution.
the idea:
Then I discovered quadtrees, and realized that my idea sucked.
Still, I would need to store coordinate points persistently on disk but still be able to query them somewhat quickly.
Luckily, I found MongoDB's geospatial queries before attempting to serialize quadtrees myself...
I discovered these geospatial querying solutions embarrassingly late, which required a lot of rewriting of features that had to do with geolocation, like placing a note and sending a chat message.
The realization that hours of work was wasted... it sucks!
How did I implement things like:
"a note was just placed at (x,y) so tell all clients whose phone screens can see the point (x,y) that there's a new note there"
before I discovered these geospatial tools?
It was a convoluted method of putting clients inside socket.io rooms representing certain geographic bounds, then broadcasting events to rooms whose bounds overlapped with the coordinate point where the event happened.
In the end, I've learned my lesson. That is, chances are, there's already a library for it.
premature optimization. i fell for it.
This optimization idea I had was amazingly tempting. I had no performance problems thus far, but...
Just think: users who query for data within a geographic rectangle are likely to query for data in adjacent areas soon after (as they pan around the map, zoom in/out).
After receiving a request, why not do the big database query for data points just outside this rectangle beforehand and cache them in memory somewhere, in anticipation for the subsequent request on the adjacent area?
How exciting. Aren't you excited?
The (perceived) need for a cache for my database was born, and I became another victim to Redis and their favorite buzzwords like "fast", "scalable", and "in-memory cache".
One week later, there was no perceivable difference in geoquery time, and it now took 50x longer to add a new post to the database. ✨
this decreased to ~2-3x after I discovered Redis's pipelining feature, but that will never make up for the time I've lost...
literally who decided latitude goes first
The convention of writing latitude before longitude (it's even ISO standard) despite latitude being the "y" axis admittedly caused me many bugs before I finally learned my lesson.
latitude is like a ladder
This phrase used to remember latitude/longitude messed me up so much.
"latitude lines run across the globe like the rungs of a ladder"
The rungs of a ladder go side to side, so I kept thinking that latitude itself meant "horizontal"