More Projects
59 in total
Helix Registration & Mail Backend
Creative Computing Society · 2024
The backend that ran registration, the check-in desk and every mail-out for a season of society events — HackTU, Helix, Vortex and the competitive-programming contests. Seven Mongoose models cover team and per-member registration, idea submission with link validation, shortlisting and waitlisting, live QR check-in and lazy table allocation assigned only when a team's first member actually arrives, with a Flutter scanner on the volunteers' phones. The same service carries the comms calendar: fifteen hand-written Outlook-safe HTML templates, spreadsheet import and export, and cohort sends throttled to roughly one invocation every two seconds because the whole thing runs off one institutional Gmail account with no sending provider behind it. The sleeps pace the calls rather than the deliveries — `sendMail` is never actually awaited, so a rejected send surfaces only as a log line.
Project Details
Creative Computing Society
2024
Society · Backend
Registration is the easy half. The hard half is the desk at 09:30 on the morning of the hackathon, where several hundred people arrive at once, half of them have a teammate who dropped out, and every one of them needs to be matched to a team, a table and a wristband before the opening ceremony starts. This is the backend that ran that desk for HackTU and the wider Helix festival, and, out of the same process, every announcement, shortlist and check-in pass that got people to it.
Why the queue is the design problem
Everything about this system is downstream of one number: how long a single participant spends at the desk. At four hundred people and three volunteers, ten seconds each is twenty minutes of queue and thirty seconds each is an hour, and the ceremony does not move. So every interaction had to collapse to one action. Scan, see the name and the table, tap present. No searching, no typing, no asking the participant which team they are on.
That is why the member is the unit of identity rather than the team. Each member gets their own document with its own ObjectId, that id is what the pre-event mail encodes into a QR, and that QR is the only thing the desk needs. The team document also keeps an embedded copy of the member list, so rendering a team costs one read - deliberate duplication, because at the desk a stale field is survivable and a second round trip is not.
What it covers
- Multi-event registration. Separate models and endpoints for the hackathon plus the Vortex data-science contest, the competitive-programming session and speaker-session signups - one service rather than four throwaway forms. Seven Mongoose models in total.
- Team registration and lookup.
hacktu/registervalidates every member's email and ten-digit phone, refuses the team if any member's email already exists anywhere in the collection, then creates the team and one document per member and mails each of them a confirmation.hacktu/findTeamandhacktu/verifyTeamlet a leader pull up an existing entry and confirm it by matching an email and phone against the embedded roster. - Checkpoint Zero.
hacktu/submitIdearecords the idea title, description, category and a video link, with a uniqueness constraint on the link and a validator that checks it is a genuine Google Drive or YouTube URL before accepting it - because a broken submission link discovered during judging is unrecoverable. The same validator set covers institute roll numbers, ten-digit phone numbers and emails. - Shortlisting and waitlisting. Admin routes to select, reject or waitlist a team, with
selected,lockedandwaitlistflags on the team document and a shortlist board reading off them. - Live check-in.
markpresentandmarkabsentflip a per-membercheckedInboolean and stamp the time, while the team document keeps acheckedInCountso the desk can see at a glance that four of five have arrived. AliveCheckInendpoint feeds the running count. - Table allocation, done lazily. A separate table model maps each team to a numbered desk, and the number is assigned at the moment the team's first member checks in rather than at registration. No-shows therefore never burn a desk, and the hall fills in arrival order. Thapar teams number upward from 2, visiting colleges from 36, with a blacklist of table numbers that do not exist in the room and a jump from 34 to 57 across a hall boundary.
- Roster surgery. Add, edit and delete members, delete whole teams - because on the day, teams merge and members swap, and the desk needs to fix it without a database client.
- Equipment tracking.
takeExtentionandreturnExtentionflags on the team, because 150 power extension cords go out and you would like most of them back. - Spreadsheet in and out.
register.from.excel.jsreads a shortlist workbook withxlsx, converts the first sheet to JSON and mails each row;make.excel.jsjoins the team and member models and writes an ExcelJS workbook the organisers can hand to the office, filtered to shortlisted teams from outside colleges - the list the institute actually asks for. - Request logging to a file plus
express-devicecapture andtrust proxy, so a check-in dispute has an audit trail. The database connection handler mails me directly if Mongo refuses the connection, which is the alerting you build when the alerting budget is zero.
The desk-side companion
The QR mails are only useful if something can read them, so there is a Flutter scanner app running on
the volunteers' phones. It locks behind a four-digit passkey, scans through flutter_barcode_scanner,
posts the scanned member id to admin/hacktu/getUserData, and shows the volunteer the participant's
name, team, college and table number with present and absent actions. There is also a numeric-keypad
fallback for entering a roll number directly, for the inevitable phone with a dead screen. Every call
carries a shared admin password in the JSON body, which is the whole authorisation story: the same
literal string is compared inside each admin handler and baked into the app.
The mail half
The constraint that shaped the other half of this service is that there was no budget for a sending
provider. No SendGrid, no SES, no Mailgun. One institutional Gmail account, ccs@thapar.edu, with a
sixteen-character app password and a daily cap somewhere in the low thousands. The throttling, the
manual campaign selection, the resume offsets and the choice to blast the institute-wide alias for
the big announcements rather than iterating five thousand addresses all follow from that. A few
thousand messages went out this way across HackTU, Helix, Vortex and the CP contests.
Fifteen HTML templates across sixteen exported functions carry the comms calendar: HackTU announcements and offline registration, Helix announcements, Vortex instructions and last-day reminders, CP contest instructions, last-day and finalist notices, online-mode shortlists, waitlist notices, WhatsApp group invites and the QR check-in pass every participant carries in on their phone. They are parameterised rather than static - the finalist template takes a name, a reporting time and a venue, so one function produces every personalised shortlist mail - and the cohort senders above them pull whole participant collections, CP contestants, Vortex entrants, and HackTU teams with and without a college affiliation. Every message carries two faculty addresses on the cc line, which is how the department kept a record of what students were sent.
Those templates are most of the codebase: about seventeen thousand lines against four hundred and
seventy-seven lines of everything else. The logic is small and the markup is enormous, because HTML
email is enormous. Each template opens with an XHTML 1.0 Transitional doctype and three Microsoft
Office namespace declarations, uses VML <v:background> and <v:roundrect> for the full-bleed
header image and the rounded call-to-action button that Outlook's Word rendering engine refuses to do
in CSS, carries between fifty and eighty-six mso conditional blocks, and nests tables three to five
deep at a median of eighteen <table> elements per file. The check-in pass is the one that looks
clever and is not: rather than generating a QR server side and attaching it, the template embeds a
remote image pointed at a public QR service with the pass payload in the query string. It renders in
Gmail, and it has two problems worth naming - the third-party service sees every attendee's token,
and desktop clients that block remote images render a blank pass.
Two practical details mattered more than any of that. Gmail will cut you off, so the cohort loops
sleep a second before each send and another second after, which moves the service at roughly one
recipient every two seconds and makes a large cohort take hours, while the spreadsheet importer runs
a shorter 500 ms and 1000 ms pair. Slow was the correct answer; the alternative was a suspended
account mid-campaign. And console.log is monkeypatched at startup to prefix an ISO timestamp and
tee every line into logs/logs.txt, with each send logging the recipient and a running counter -
which is why several loops start at a hardcoded index rather than zero, one at 19, one at 500, one at
10, the offsets left in place after a campaign died partway and was restarted from the log. There is
no job store, no send-state persistence and no idempotency key. The queue is the for loop and the
recovery mechanism is editing its initialiser.
The design flaw follows from the same shape: campaigns are not routes. start_routes() is a stack of
commented-out one-shot sendMail calls, eleven of them plus seven commented bulk drivers, and to run
one you uncomment the line, restart the server, watch the log, then comment it back. Three of those
lines target students@thapar.edu, the entire student alias. There is no dry run, no scheduling, and
nothing preventing a double send except remembering to re-comment. The deeper flaw is one level down.
sendMail builds a transporter, defines an inner async function, and calls it without awaiting -
main().catch(console.error). So every await sendMail(...) at every call site resolves
immediately, the sleeps throttle invocation rather than completion, and the try/catch blocks
wrapped around the sends are structurally unable to catch a delivery failure. A rejected send
surfaces only as a line in the log, and a new transporter is constructed per message, so there is no
connection reuse either.
The lineage is short in both directions. The ancestor is Bulk Mailer, three months older: one file,
one hardcoded subject line, one recipient, no loop, no database, no throttle, and a 1,234-line Stripo
export sitting beside it as a loose HTML file the script never reads. The step from there to here is
the step from a loose HTML file to a parameterised module returning a template literal, and from a
single send to a database-driven throttled loop - though curiously the older one created its
transporter once at module scope and this rewrite regressed that. The step after is the argument the
flaws above make for themselves: a persisted queue with a worker that retries, respects a rate limit
it can observe rather than guess at, and can be inspected mid-campaign. That is what the later Sat
Mailer became.
Where it would break
The lazy table allocator computes the blacklist of missing table numbers and then overwrites the
result on the next line with the last allocated number, so the skip list never actually applies. The
check-in counter is a read-modify-write of checkedInCount + 1, so two volunteers scanning two
members of the same team at the same instant lose an increment - it wants an atomic $inc. And the
Mongoose update that follows returns the pre-update document by default, which means the comparison
that flips a team's overall checkedIn flag is testing a stale count. None of that surfaced on the
day, because a desk is naturally serialised by the length of the queue, which is exactly the kind of
bug an event papers over and production would not.
Deployment and the state it was left in
Node 18 on a slim Debian base, one Compose service on port 4378, the logs directory bind-mounted out so a volunteer's report can be checked against the file. HTTPS was set up and then commented out in favour of terminating elsewhere.
The repository is a snapshot of an operational tool rather than a maintained service, and it shows.
The route table itself is commented out in app.js - the mount of /api/v1 and the require above it
- because after the event the process was being restarted only to fire one-off mail jobs, and boot still calls a spreadsheet mailer. The commented-out campaign stack sits directly above that mount, every announcement and nudge and finalist letter left in the order it was sent. It is an ugly artefact and an honest one.
Two more admissions belong on the list. The credentials live in a plaintext config.env next to the
code. And the log directory, which contains every recipient address the service has ever touched, is
served as a static route.
Project Details
Creative Computing Society
2024
Society · Backend