Intro
I have a project that I wrote 7 years ago, its still running on firebase.. no hickups (I had to occusionally update to a new node lts environment).. but it still worsks, at the time of writing/ implementing, ratelimiter was not implemented, So everyonce in a while I see the request logs and I see some IPs hitting the server with 1000s of requests, judging from the rquest.. its easy to distinguish them as bots and scanners. I have to weed them out, but I have to write it in a way that will not use unnecessary firestore reads and writes. (yea I also used firestore for this probject)
Constraints before going in
I have to make sure the firestore reads and writes are minimal, one read + one write per request, make sure automatic TTL purging keeps the firestore collection small.
I have no Redis / Memcached layer here, so I can’t use them..(using them would be the best solution)
Schema thats need to be changed in firestore
A new collection called ip_throttle, we will use the document ID as the client IP address.
ip_throttle/
└─ {ipAddress} # document ID == client IP
• count number # requests in current window
• windowStart timestamp # start of the rolling window
• expiresAt timestamp # TTL field (auto‑deleted)
We can leverage the Firestore TTL to automatically delete documents after the marked time, I have to mark my field expiresAt in the TTL policy.
Configurations needed
Follwing new configurations are needed, these doen’t involve code changes, In 5 minutes, we have 100 requests per IP.
firebase functions:config:set \
ratelimit.window_seconds=3600 \
ratelimit.max_hits=100
For local emulation we can utilise
firebase functions:config:get > .runtimeconfig.json
Implementing.
Nowadsys I dont care if the project was in TS or not, I just choose js and go with it, back then I was still using TS, so I have to write the code in TS.
// abstraction to Firestore
const db = getConnection();
export async function rateLimiter(
ip: string,
windowSeconds = 3600, // just incase config does not exist
maxHits = 100 // just incase config does not exist
): Promise<boolean> {
const now = Timestamp.now();
const ref = db.collection("ip_throttle").doc(ip);
return db.runTransaction(async (tx: Transaction) => {
const snap = await tx.get(ref);
// this is the first request we get from this IP
if (!snap.exists) {
tx.set(ref, {
count: 1,
windowStart: now,
expiresAt: Timestamp.fromMillis(
now.toMillis() + windowSeconds * 1000
),
});
logger.info(`IP ${ip}: first hit, record created.`);
return true;
}
const data = snap.data()!;
const ageMs = now.toMillis() - data.windowStart.toMillis();
// if the window has expired, reset it
if (ageMs > windowSeconds * 1000) {
tx.set(
ref,
{
count: 1,
windowStart: now,
expiresAt: Timestamp.fromMillis(
now.toMillis() + windowSeconds * 1000
),
},
{ merge: true }
);
logger.info(`IP ${ip}: window reset.`);
return true;
}
// if the request is within the window and within limit.
if (data.count < maxHits) {
tx.update(ref, { count: FieldValue.increment(1) });
logger.debug(`IP ${ip}: allowed (${data.count + 1}/${maxHits}).`);
return true;
}
// if the request is over the limit
logger.warn(
`IP ${ip}: throttled (${data.count}/${maxHits} in ${windowSeconds}s).`
);
return false;
});
}
Now to configure in router and sending out the response
// existing code
const cfg = process.env;
const windowSeconds =
Number(cfg.RL_WINDOW_SECONDS) ||
Number(cfg.RATELIMIT_WINDOW_SECONDS) ||
3600;
const maxHits =
Number(cfg.RL_MAX_HITS) ||
Number(cfg.RATELIMIT_MAX_HITS) ||
5;
// check for ratelimit
if (!(await rateLimiter(ip, windowSeconds, maxHits))) {
res.status(429).json({ error: "Too many requests. Try again later." });
return;
}
// original code
In action in production
Conclusion
The solution I’ve implemented is simple, effective and horizonally scalable all within firebase. No extra services, minimal reads and writes. I was able to implement and deploy to production in one single sitting. I had validated this with ai thinking models to find downsides, so it gave me a motivation to complete it quickly.
Personal updates
Like with tradition, I have the personal updates here, I moved the personal section from top to the end of the post over the years. I wana write/ share soo much more.. remind myself more. Been travelling back and forth from work location & home, Been busy with work a lot, though I shouldn’t take it too seriously, but when its a new skill, im all up for it, even if its people or project management, the one thing that pains is that there is a onshore team and sync with them is a pain (currently at the time of writing this you work in VZ), evey week, every months feels like an instant, its monday.. then its already friday, then its friday night start.. then its sunday night again with an instant. Feels like a cycle, The older I get, time does feel to go a lot faster. I even stopped playing the occasional gaming on the side.
My timer tracking project (didnt make it open source yet) was a lot helpful over the months, but I think im lossing a grip on it, its too micro managementy, everytime I do something, I have to open and start and end it, feels like..its taking lot of time just to do that between each teask. So what I’ll do is stop using the time tracking app that I built for 1 or 2 months and see how I fare, If I can still be productive without it and can remeber and complete tasks, then its great..future me how is it going?
Personally I have commitments now and that too is a time sink, Im not keeping tabs on how much time Im spending, I think to myself on heindsight at the end of the week or during self refelction, small small actions, small pieces of time each day over the week.. adding up.. is a lot of time. but I can’t ignore it can I, now I know why my friends’ changed behaviours is attributed to. Its nobody’s fault, thats just life.
The AI slop is all over the place with the internet, its the worst. Manytimes I wonder.. I think for some/ many tasks, If people didn’t just overrely on AI, they can complete the job/ task way faster. Again on some tasks, its way easier with AI and reduces lot of time.
Another trend Im noticing is, people are too much relient on AI to write a feature in a product or entire codebase/product. When something comes up later on the project, they don’t know howt to fix it, I’ve seen it inside office projects (migration of entire codebase with AI and the people sturggle later), on the internet and word of mouth. Future me.. how its it going?
About the thing on may.. I’d be lying If I said its good/ going great. It is what it is, Trust on the process, I have to keep doing what I do, keep going, keep moving forward, things will fall in place, For now.. For this moment, I know, I have to make the right decision (Future me remember?) and the right decision I know is to stay without making any rash decisions..me reading..what happpended?..hope I/we are in a better place then..when are you reading this? 20’s, 30’s, 40’s?.