Helix Registration & Mail Backend
Archived
Society · Backend

Helix Registration & Mail Backend

Creative Computing Society · 2024

The backend that ran registration, the check-in desk and every mail-out for a season of society events — HackTU, Helix, Vortex and the competitive-programming contests. Seven Mongoose models cover team and per-member registration, idea submission with link validation, shortlisting and waitlisting, live QR check-in and lazy table allocation assigned only when a team's first member actually arrives, with a Flutter scanner on the volunteers' phones. The same service carries the comms calendar: fifteen hand-written Outlook-safe HTML templates, spreadsheet import and export, and cohort sends throttled to roughly one invocation every two seconds because the whole thing runs off one institutional Gmail account with no sending provider behind it. The sleeps pace the calls rather than the deliveries — `sendMail` is never actually awaited, so a rejected send surfaces only as a log line.

Built with
Node.jsNode.js
ExpressExpress
MongoDBMongoDB
MongooseMongoose
NodemailerNodemailer
ExcelJSExcelJS
xlsxxlsx
FlutterFlutter
DartDart
DockerDocker
Project Details

STATUS
Archived
ORGANISATION

Creative Computing Society

YEAR

2024

TYPE

Society · Backend

TAGS
Events
Registration
Email
Automation

Registration is the easy half. The hard half is the desk at 09:30 on the morning of the hackathon, where several hundred people arrive at once, half of them have a teammate who dropped out, and every one of them needs to be matched to a team, a table and a wristband before the opening ceremony starts. This is the backend that ran that desk for HackTU and the wider Helix festival, and, out of the same process, every announcement, shortlist and check-in pass that got people to it.

Why the queue is the design problem

Everything about this system is downstream of one number: how long a single participant spends at the desk. At four hundred people and three volunteers, ten seconds each is twenty minutes of queue and thirty seconds each is an hour, and the ceremony does not move. So every interaction had to collapse to one action. Scan, see the name and the table, tap present. No searching, no typing, no asking the participant which team they are on.

That is why the member is the unit of identity rather than the team. Each member gets their own document with its own ObjectId, that id is what the pre-event mail encodes into a QR, and that QR is the only thing the desk needs. The team document also keeps an embedded copy of the member list, so rendering a team costs one read - deliberate duplication, because at the desk a stale field is survivable and a second round trip is not.

What it covers

  • Multi-event registration. Separate models and endpoints for the hackathon plus the Vortex data-science contest, the competitive-programming session and speaker-session signups - one service rather than four throwaway forms. Seven Mongoose models in total.
  • Team registration and lookup. hacktu/register validates every member's email and ten-digit phone, refuses the team if any member's email already exists anywhere in the collection, then creates the team and one document per member and mails each of them a confirmation. hacktu/findTeam and hacktu/verifyTeam let a leader pull up an existing entry and confirm it by matching an email and phone against the embedded roster.
  • Checkpoint Zero. hacktu/submitIdea records the idea title, description, category and a video link, with a uniqueness constraint on the link and a validator that checks it is a genuine Google Drive or YouTube URL before accepting it - because a broken submission link discovered during judging is unrecoverable. The same validator set covers institute roll numbers, ten-digit phone numbers and emails.
  • Shortlisting and waitlisting. Admin routes to select, reject or waitlist a team, with selected, locked and waitlist flags on the team document and a shortlist board reading off them.
  • Live check-in. markpresent and markabsent flip a per-member checkedIn boolean and stamp the time, while the team document keeps a checkedInCount so the desk can see at a glance that four of five have arrived. A liveCheckIn endpoint feeds the running count.
  • Table allocation, done lazily. A separate table model maps each team to a numbered desk, and the number is assigned at the moment the team's first member checks in rather than at registration. No-shows therefore never burn a desk, and the hall fills in arrival order. Thapar teams number upward from 2, visiting colleges from 36, with a blacklist of table numbers that do not exist in the room and a jump from 34 to 57 across a hall boundary.
  • Roster surgery. Add, edit and delete members, delete whole teams - because on the day, teams merge and members swap, and the desk needs to fix it without a database client.
  • Equipment tracking. takeExtention and returnExtention flags on the team, because 150 power extension cords go out and you would like most of them back.
  • Spreadsheet in and out. register.from.excel.js reads a shortlist workbook with xlsx, converts the first sheet to JSON and mails each row; make.excel.js joins the team and member models and writes an ExcelJS workbook the organisers can hand to the office, filtered to shortlisted teams from outside colleges - the list the institute actually asks for.
  • Request logging to a file plus express-device capture and trust proxy, so a check-in dispute has an audit trail. The database connection handler mails me directly if Mongo refuses the connection, which is the alerting you build when the alerting budget is zero.

The desk-side companion

The QR mails are only useful if something can read them, so there is a Flutter scanner app running on the volunteers' phones. It locks behind a four-digit passkey, scans through flutter_barcode_scanner, posts the scanned member id to admin/hacktu/getUserData, and shows the volunteer the participant's name, team, college and table number with present and absent actions. There is also a numeric-keypad fallback for entering a roll number directly, for the inevitable phone with a dead screen. Every call carries a shared admin password in the JSON body, which is the whole authorisation story: the same literal string is compared inside each admin handler and baked into the app.

The mail half

The constraint that shaped the other half of this service is that there was no budget for a sending provider. No SendGrid, no SES, no Mailgun. One institutional Gmail account, ccs@thapar.edu, with a sixteen-character app password and a daily cap somewhere in the low thousands. The throttling, the manual campaign selection, the resume offsets and the choice to blast the institute-wide alias for the big announcements rather than iterating five thousand addresses all follow from that. A few thousand messages went out this way across HackTU, Helix, Vortex and the CP contests.

Fifteen HTML templates across sixteen exported functions carry the comms calendar: HackTU announcements and offline registration, Helix announcements, Vortex instructions and last-day reminders, CP contest instructions, last-day and finalist notices, online-mode shortlists, waitlist notices, WhatsApp group invites and the QR check-in pass every participant carries in on their phone. They are parameterised rather than static - the finalist template takes a name, a reporting time and a venue, so one function produces every personalised shortlist mail - and the cohort senders above them pull whole participant collections, CP contestants, Vortex entrants, and HackTU teams with and without a college affiliation. Every message carries two faculty addresses on the cc line, which is how the department kept a record of what students were sent.

Those templates are most of the codebase: about seventeen thousand lines against four hundred and seventy-seven lines of everything else. The logic is small and the markup is enormous, because HTML email is enormous. Each template opens with an XHTML 1.0 Transitional doctype and three Microsoft Office namespace declarations, uses VML <v:background> and <v:roundrect> for the full-bleed header image and the rounded call-to-action button that Outlook's Word rendering engine refuses to do in CSS, carries between fifty and eighty-six mso conditional blocks, and nests tables three to five deep at a median of eighteen <table> elements per file. The check-in pass is the one that looks clever and is not: rather than generating a QR server side and attaching it, the template embeds a remote image pointed at a public QR service with the pass payload in the query string. It renders in Gmail, and it has two problems worth naming - the third-party service sees every attendee's token, and desktop clients that block remote images render a blank pass.

Two practical details mattered more than any of that. Gmail will cut you off, so the cohort loops sleep a second before each send and another second after, which moves the service at roughly one recipient every two seconds and makes a large cohort take hours, while the spreadsheet importer runs a shorter 500 ms and 1000 ms pair. Slow was the correct answer; the alternative was a suspended account mid-campaign. And console.log is monkeypatched at startup to prefix an ISO timestamp and tee every line into logs/logs.txt, with each send logging the recipient and a running counter - which is why several loops start at a hardcoded index rather than zero, one at 19, one at 500, one at 10, the offsets left in place after a campaign died partway and was restarted from the log. There is no job store, no send-state persistence and no idempotency key. The queue is the for loop and the recovery mechanism is editing its initialiser.

The design flaw follows from the same shape: campaigns are not routes. start_routes() is a stack of commented-out one-shot sendMail calls, eleven of them plus seven commented bulk drivers, and to run one you uncomment the line, restart the server, watch the log, then comment it back. Three of those lines target students@thapar.edu, the entire student alias. There is no dry run, no scheduling, and nothing preventing a double send except remembering to re-comment. The deeper flaw is one level down. sendMail builds a transporter, defines an inner async function, and calls it without awaiting - main().catch(console.error). So every await sendMail(...) at every call site resolves immediately, the sleeps throttle invocation rather than completion, and the try/catch blocks wrapped around the sends are structurally unable to catch a delivery failure. A rejected send surfaces only as a line in the log, and a new transporter is constructed per message, so there is no connection reuse either.

The lineage is short in both directions. The ancestor is Bulk Mailer, three months older: one file, one hardcoded subject line, one recipient, no loop, no database, no throttle, and a 1,234-line Stripo export sitting beside it as a loose HTML file the script never reads. The step from there to here is the step from a loose HTML file to a parameterised module returning a template literal, and from a single send to a database-driven throttled loop - though curiously the older one created its transporter once at module scope and this rewrite regressed that. The step after is the argument the flaws above make for themselves: a persisted queue with a worker that retries, respects a rate limit it can observe rather than guess at, and can be inspected mid-campaign. That is what the later Sat Mailer became.

Where it would break

The lazy table allocator computes the blacklist of missing table numbers and then overwrites the result on the next line with the last allocated number, so the skip list never actually applies. The check-in counter is a read-modify-write of checkedInCount + 1, so two volunteers scanning two members of the same team at the same instant lose an increment - it wants an atomic $inc. And the Mongoose update that follows returns the pre-update document by default, which means the comparison that flips a team's overall checkedIn flag is testing a stale count. None of that surfaced on the day, because a desk is naturally serialised by the length of the queue, which is exactly the kind of bug an event papers over and production would not.

Deployment and the state it was left in

Node 18 on a slim Debian base, one Compose service on port 4378, the logs directory bind-mounted out so a volunteer's report can be checked against the file. HTTPS was set up and then commented out in favour of terminating elsewhere.

The repository is a snapshot of an operational tool rather than a maintained service, and it shows. The route table itself is commented out in app.js - the mount of /api/v1 and the require above it

  • because after the event the process was being restarted only to fire one-off mail jobs, and boot still calls a spreadsheet mailer. The commented-out campaign stack sits directly above that mount, every announcement and nudge and finalist letter left in the order it was sent. It is an ugly artefact and an honest one.

Two more admissions belong on the list. The credentials live in a plaintext config.env next to the code. And the log directory, which contains every recipient address the service has ever touched, is served as a static route.

Project Details

STATUS
Archived
ORGANISATION

Creative Computing Society

YEAR

2024

TYPE

Society · Backend

TAGS
Events
Registration
Email
Automation