<p>If you're using a full-text search with MongoDB and filtering out soft-deleted or duplicate records using fields like <code>removed: { $exists: false }</code> and <code>dupSlug: { $exists: false }</code>, you might wonder: <i>should I index those fields for performance?</i></p><figure class="image"><img src="https://images.unsplash.com/photo-1742975710558-1caf75cdbda2?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHwxMzJ8fGluZGV4fGVufDB8fHx8MTc1MTY2MDk0NXww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@yanahd?utm_source=DNX&amp;utm_medium=referral">Allen Y</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>🚫 The Short Answer: <strong>No, you don't need to.</strong></h3><h3>Why?</h3><p><strong>MongoDB applies </strong><code><strong>$text</strong></code><strong> search first</strong>, using your defined text index (e.g., <code>SearchSwimmerByNameTeamLsc</code>). This is handled efficiently with <code>IXSCAN</code> and <code>TEXT_MATCH</code>.</p><p>Filters like <code>removed</code> and <code>dupSlug</code> are applied <strong>later</strong>, during the <code>FETCH</code> stage. At that point:</p><p>MongoDB has already narrowed down the candidates using the text index.</p><p>It only needs to examine a small number of documents — for example, around ~2,865 in our case — which is very fast.</p><p><strong>Indexing </strong><code><strong>removed</strong></code><strong> or </strong><code><strong>dupSlug</strong></code><strong> won’t help</strong>, because:</p><p><code>$exists: false</code> conditions <strong>cannot use sparse indexes</strong> (they only work with <code>$exists: true</code>).</p><p>MongoDB <strong>doesn't allow combining </strong><code><strong>$text</strong></code><strong> with other fields</strong> in compound indexes.</p><p>You could create a partial index, but that only helps if you're querying those fields <strong>independently of text search</strong>.</p><h3>✅ Summary</h3><p>Your query is already efficient and does <strong>not require additional indexes</strong> on <code>removed</code> or <code>dupSlug</code>.</p><p>Indexing them would add storage and write overhead with <strong>no gain</strong> in performance.</p><p>Only consider indexing if:</p><p>You start querying these fields <strong>without full-text search</strong>, or</p><p>You refactor your logic to use something like <code>isActive: true</code> (which is easier to index).</p><p>Still seeing performance issues? You might benefit from replacing <code>$exists: false</code> with a flag field (<code>isDeleted</code>, <code>isDuplicate</code>) that defaults to <code>true</code> and can be indexed cleanly.</p><p>Let me know if you want to explore that option or review your actual query plans.</p>

If you're using a full-text search with MongoDB and filtering out soft-deleted or duplicate records using fields like removed: { $exists: false } and dupSlug: { $exists: false }, you might wonder: should I index those fields for performance?

🧠 Should You Index removed and dupSlug in MongoDB? Probably Not.

<p>If you've ever worked with user-generated data (like swim meet names, usernames, or hashtags), you've likely run into this:</p><blockquote><p><strong>A single long word with no spaces breaks your layout.</strong></p></blockquote><p>For example:</p><pre><code class="language-plaintext">The2025NSCAgeGroupChampionshipsPresentedBySwimStandardsDotCom
</code></pre><p>This kind of string won't wrap naturally — and it might stretch your card, table, or even the entire mobile page.</p><figure class="image"><img src="https://images.unsplash.com/photo-1470241918028-bc2b588e1510?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHwxMDV8fGJyZWFrfGVufDB8fHx8MTc1MTY2MDc4NHww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@elevatedtv?utm_source=DNX&amp;utm_medium=referral">Ryan Pohanic</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Let’s fix that.</p><h3>🧵 The Problem</h3><p>By default, browsers only break lines:</p><p>at spaces,</p><p>hyphens (<code>-</code>),</p><p>or where language rules allow.</p><p>So a long, space-free meet name inside a container like this:</p><pre><code class="language-html">&lt;div class="meet-name"&gt;
  The2025NSCAgeGroupChampionshipsPresentedBySwimStandardsDotCom
&lt;/div&gt;
</code></pre><p>...will overflow, stretch, or get clipped.</p><h3>✅ The Solution: <code>overflow-wrap: anywhere</code></h3><p>To make the string wrap <strong>even in the middle of a word</strong>, use:</p><pre><code class="language-css">.meet-name {
  overflow-wrap: anywhere;
  word-break: normal; /* optional, for clarity */
}
</code></pre><h3>🔍 Why This Works</h3><p><code>overflow-wrap: anywhere</code> tells the browser:</p><blockquote><p>"Wrap at normal places (like spaces) if possible, <strong>but if not, break wherever you must</strong> to avoid overflow."</p></blockquote><p>It won’t break every word unnecessarily — only when there's no good break point. So regular short names like <code>Summer Champs 2025</code> won’t be affected.</p><h3>🧪 Example</h3><h4>Before:</h4><pre><code class="language-css">.meet-name {
  overflow-wrap: normal;
}
</code></pre><p>Result: long name explodes the layout.</p><h4>After:</h4><pre><code class="language-css">.meet-name {
  overflow-wrap: anywhere;
}
</code></pre><p>Result: the long name breaks cleanly and respects the container width.</p><h3>📱 Mobile Friendly</h3><p>This is especially helpful for:</p><p><strong>mobile layouts</strong> with limited width</p><p><strong>tables and cards</strong> where overflow ruins spacing</p><p><strong>search results</strong> with unpredictable meet names</p><h3>✨ Bonus Tip</h3><p>You can combine with:</p><pre><code class="language-css">white-space: pre-wrap;
</code></pre><p>If you want to respect line breaks (<code>\n</code>) and still wrap long text.</p><h3>🏁 Final Thoughts</h3><p>If your app handles swim meets, usernames, or imported data, you’re bound to hit long, space-free strings.<br>Don’t let them break your UI — let CSS handle it smartly with:</p><pre><code class="language-css">overflow-wrap: anywhere;
</code></pre><p>It’s a small fix that saves big headaches.</p><p>Let me know if you want this styled for your own site or turned into a reusable CSS class like <code>.wrap-anywhere</code>.</p>

If you're dealing with long, unbreakable strings like swim meet names that have no spaces, overflow-wrap: anywhere is a simple CSS fix that ensures your layout stays intact. It tells the browser to break text at normal points first, but if no breaks are found, it will force a break anywhere to prevent overflow. This is especially useful for mobile layouts, tables, or cards where long names can otherwise stretch or break the design.

💥 How to Handle Long Meet Names with overflow-wrap: anywhere

<p>For nearly <strong>three years</strong>, SwimStandards.com has quietly had a problem that I didn’t realize until recently: <strong>Googlebot was failing to crawl </strong><code><strong>www.swimstandards.com</strong></code> due to an SSL error.</p><figure class="image"><img src="https://images.unsplash.com/photo-1614036417651-efe5912149d8?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw4NHx8Y2VydGlmaWNhdGV8ZW58MHx8fHwxNzUwNjMzNDUzfDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@brandsandpeople?utm_source=DNX&amp;utm_medium=referral">Brands&amp;People</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>I always assumed everything was working fine because:</p><p>I only use <code>https://swimstandards.com</code></p><p>Google Search Console didn’t report any errors for the root domain</p><p>I never intentionally promoted the <code>www</code> version</p><p>But last week, I finally looked into a persistent <strong>“High fail rate”</strong> warning from Google Search Console — and I found the root cause.</p><h3>What Happened?</h3><p>When I first set up SSL using Let's Encrypt, I only issued a certificate for:</p><pre><code class="language-plaintext">swimstandards.com</code></pre><p>I didn’t realize that <code><strong>www.swimstandards.com</strong></code><strong> was not included</strong> — and that matters.</p><p>Googlebot (and some users) occasionally try the <code>www</code> version. Since the SSL certificate didn’t cover it, requests to <code>https://www.swimstandards.com</code> <strong>failed with a certificate mismatch</strong>, leading to crawl failures and potentially missed indexing.</p><h3>The Fix (Finally)</h3><p>I reissued the SSL certificate using Certbot with both domains:</p><pre><code class="language-plaintext">sudo certbot --apache -d swimstandards.com -d www.swimstandards.com</code></pre><p>This updated the certificate to include both:</p><pre><code class="language-plaintext">DNS:swimstandards.com
DNS:www.swimstandards.com</code></pre><p>I also added a redirect so that any requests to <code>www.swimstandards.com</code> get properly forwarded to the canonical root domain:</p><pre><code class="language-plaintext">https://swimstandards.com</code></pre><h3>Did This Affect SEO?</h3><p>Honestly — probably <strong>not much</strong>, but it's possible:</p><p>Google favors a consistent, canonical domain structure.</p><p>Failing SSL on <code>www</code> might have prevented some backlinks from being credited correctly.</p><p>Googlebot crawl errors, even if minor, aren’t ideal over a long period.</p><p>If there was any SEO impact, it was likely small, but avoidable.</p><h3>Lesson Learned</h3><p>If you're running a site:</p><p>Always <strong>include both </strong><code><strong>domain.com</strong></code><strong> and </strong><code><strong>www.domain.com</strong></code> when issuing your SSL certificate.</p><p>Always set up <strong>proper redirects</strong> from the non-canonical domain.</p><p>Use tools like <code>curl</code>, <code>openssl</code>, and <a target="_blank" rel="noopener noreferrer" href="https://www.ssllabs.com/ssltest/">SSL Labs</a> to test both versions.</p><p>Check Search Console regularly — <strong>even warnings you think don’t matter</strong>.</p><h3>TL;DR</h3><p>For 3 years, <code>https://www.swimstandards.com</code> was broken due to an SSL oversight. I fixed it in 10 minutes — and I wish I had done it sooner.</p>

For nearly three years, SwimStandards.com has quietly had a problem that I didn’t realize until recently: Googlebot was failing to crawl www.swimstandards.com due to an SSL error.
I always assumed everything was working fine because:
I only use https://swimstandards.com
Google Search Console didn’t report any errors for the root domain

Google Was Failing to Crawl My Site — Because My SSL Didn’t Cover www

<p>To protect against server failure, data loss, or accidental changes, I’ve automated daily backups of my web app — including both the <strong>MongoDB database</strong> and <strong>NodeBB forum files</strong> — and synced them securely to <strong>Google Drive</strong> using <a target="_blank" rel="noopener noreferrer" href="https://rclone.org/"><code>rclone</code></a>.</p><figure class="image"><img src="https://images.unsplash.com/photo-1624895608078-e9f564cbe3fa?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw0fHxiYWNrJTIwdXB8ZW58MHx8fHwxNzQ5Nzc3Nzc4fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@siyuan_hu?utm_source=DNX&amp;utm_medium=referral">Siyuan Hu</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>🧰 Tools Used</h3><p><strong>MongoDB</strong> for database</p><p><strong>NodeBB</strong> for forum data</p><p><code>mongodump</code> for database backup</p><p><code>rclone</code> to sync backups to Google Drive</p><p><code>cron</code> to automate the task</p><h3>📁 Backup Script</h3><p>Here’s the backup script I run every night at 2 AM:</p><pre><code class="language-plaintext">#!/bin/bash
source /home/adam/.mongodump.cnf  # contains MONGODB_URI or credentials

BACKUP_DIR="/home/adam/backups"
TIMESTAMP=$(date +"%F-%H%M")
MONGO_BACKUP="$BACKUP_DIR/mongo/mongo-backup-$TIMESTAMP"
FILES_BACKUP="$BACKUP_DIR/nodebb-files/files-backup-$TIMESTAMP"

mkdir -p "$MONGO_BACKUP" "$FILES_BACKUP"

# Backup MongoDB
mongodump --out "$MONGO_BACKUP"

# Backup NodeBB uploads/configs/logs (customize as needed)
cp -r /home/adam/nodebb/public/uploads "$FILES_BACKUP/uploads"
cp -r /home/adam/nodebb/config.json "$FILES_BACKUP/config.json"
cp -r /home/adam/nodebb/logs "$FILES_BACKUP/logs"

# Clean up local backups older than 7 days
find "$BACKUP_DIR/mongo" -type d -mtime +7 -exec rm -rf {} \;
find "$BACKUP_DIR/nodebb-files" -type d -mtime +7 -exec rm -rf {} \;

# Sync to Google Drive
rclone sync "$BACKUP_DIR" gdrive:NodeBBBackups --delete-during</code></pre><h3>🔁 Set Up Cron Job</h3><p>To run the backup daily at 2:00 AM:</p><pre><code class="language-plaintext">crontab -e</code></pre><p>Add this line:</p><pre><code class="language-plaintext">0 2 * * * /home/adam/cron/backup.sh &gt;&gt; /home/adam/cron/backup.log 2&gt;&amp;1</code></pre><h3>☁️ Google Drive Setup with <code>rclone</code></h3><p>If you're starting from scratch:</p><p>Install <code>rclone</code>:</p><pre><code class="language-plaintext">curl https://rclone.org/install.sh | sudo bash</code></pre><p>Configure a Google Drive remote:</p><pre><code class="language-plaintext">rclone config</code></pre><p>Choose <code>n</code> for new remote</p><p>Name: <code>gdrive</code></p><p>Storage: <code>drive</code></p><p>Add your <strong>own client ID and secret</strong> from Google Cloud (optional but recommended)</p><p>Use <code>rclone authorize</code> if configuring from a headless server</p><p>Test with:</p><pre><code class="language-plaintext">rclone lsd gdrive:</code></pre><h3>🔐 Tip: Keep Only the Latest Cloud Backup</h3><p>To avoid using excess space on Google Drive, the sync command:</p><pre><code class="language-plaintext">rclone sync "$BACKUP_DIR" gdrive:NodeBBBackups --delete-during</code></pre><p>ensures the remote backup mirrors your local backup folder — deleting outdated files automatically.</p><h3>✅ Automate It (Optional)</h3><p>You can create a cron job:</p><pre><code class="language-plaintext" id="PMqHxqcSi">0 2 * * * /usr/bin/rclone sync /home/adam/backups gdrive:NodeBBBackups --delete-during &gt;&gt; /var/log/rclone.log 2&gt;&amp;1
</code></pre><p>This syncs your backups daily at 2:00 AM.</p>

To protect against server failure, data loss, or accidental changes, I’ve automated daily backups of my web app — including both the MongoDB database and NodeBB forum files — and synced them securely to Google Drive using rclone.

🔒 How I Back Up My Web App to Google Drive with rclone

<p>If you're running a modern web app like Next.js, React, or Vue, you've probably noticed something strange in your AdSense dashboard:</p><blockquote><p><strong>Page views are way lower than expected</strong>, but ad impressions seem fine.</p></blockquote><p>This isn’t a bug — it’s how AdSense works (or more accurately, how it doesn’t work with SPAs).</p><figure class="image"><img src="https://images.unsplash.com/photo-1451443700141-5ddb6d85a8fc?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw0N3x8U1BBfGVufDB8fHx8MTc0OTA4NzE1Mnww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@shawneewilborn?utm_source=DNX&amp;utm_medium=referral">shawnee wilborn</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>What AdSense Actually Counts</h3><figure class="table"><table class="ui table celled"><thead><tr><th>Event</th><th>Counts AdSense Page View?</th><th>Counts Ad Impression?</th></tr></thead><tbody><tr><td>Full page reload (hard refresh)</td><td>✅ Yes</td><td>✅ Yes</td></tr><tr><td>SPA route change (no reload)</td><td>❌ No</td><td>✅ Yes (if ad is rendered with <code>push({})</code>)</td></tr></tbody></table></figure><h3>Why AdSense Page Views Are Misleading</h3><p>AdSense defines a <strong>page view</strong> as:</p><blockquote><p>A full page load where at least one ad is rendered.</p></blockquote><p>In SPAs, navigating between pages <strong>does not reload the page</strong>, so AdSense doesn't increment the page view counter — even if new content is shown and new ads appear.</p><p>Meanwhile, each time an ad is rendered, it <strong>does count as an ad impression</strong>, whether or not the URL changed.</p><h3>The Real Mistake: Calling <code>adsbygoogle.push({})</code> After Route Change</h3><p>Many developers try this:</p><pre><code class="language-plaintext">router.events.on('routeChangeComplete', () =&gt; {
  (adsbygoogle = window.adsbygoogle || []).push({});
});
</code></pre><p>But this doesn't work reliably. Why?</p><p>Because by the time that push runs, the <code>&lt;ins class="adsbygoogle"&gt;</code> tag might <strong>not yet be in the DOM</strong>. So nothing renders — no ad, no impression.</p><h3>✅ The Correct Way: Push Inside the Ad Unit</h3><p>You should call <code>adsbygoogle.push({})</code> <strong>inside your ad component</strong>, once it's mounted in the DOM.</p><p>Here’s a React example:</p><pre><code class="language-plaintext">import { useEffect, useRef } from "react";

export default function AdUnit({ adClient, adSlot }) {
  const adRef = useRef(null);

  useEffect(() =&gt; {
    if (window.adsbygoogle &amp;&amp; adRef.current) {
      try {
        (window.adsbygoogle = window.adsbygoogle || []).push({});
      } catch (e) {
        console.error("AdSense push failed:", e);
      }
    }
  }, []);

  return (
    &lt;ins
      className="adsbygoogle"
      style={{ display: "block" }}
      data-ad-client={adClient}
      data-ad-slot={adSlot}
      data-ad-format="auto"
      data-full-width-responsive="true"
      ref={adRef}
    /&gt;
  );
}
</code></pre><p>This ensures:</p><p>The container is in the DOM</p><p>AdSense can properly render the unit</p><p>An <strong>ad impression is counted</strong></p><h3>So What Should You Care About?</h3><p>If you're using a SPA, <strong>ignore AdSense page views</strong>. They are nearly meaningless.</p><p>Focus on:</p><p><strong>Ad impressions</strong> (what actually earns you money)</p><p><strong>eCPM based on impressions</strong></p><p>Making sure your ads are visible and filled</p><p>And yes — your <strong>AdSense page views will always be underreported</strong> unless the user refreshes the entire site.</p><h3>Final Thoughts</h3><p>SPAs broke the traditional pageview model years ago. Unfortunately, <strong>AdSense still hasn’t caught up</strong>. If you're relying on page view metrics to evaluate performance, you’re looking at a distorted picture.</p><p>Use the metrics that matter: <strong>impressions and revenue</strong>. Just make sure your ad units mount correctly, push cleanly, and fill reliably.</p>

If you're running a modern web app like Next.js, React, or Vue, you've probably noticed something strange in your AdSense dashboard:
Page views are way lower than expected, but ad impressions seem fine.
This isn’t a bug — it’s how AdSense works (or more accurately, how it doesn’t work with SPAs).

Why Google AdSense Page Views Are Broken on SPAs (and What Actually Counts)

<p>On May 28, I analyzed one of my websites' traffic over 20 minutes to figure out whether my current 2-core setup was holding up — or holding me back. This is what I found.</p><figure class="image"><img src="https://images.unsplash.com/photo-1555617778-02518510b9fa?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw0NXx8Y29yZXxlbnwwfHx8fDE3NDg0ODA4ODF8MA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@christianw?utm_source=DNX&amp;utm_medium=referral">Christian Wiediger</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h2>🔍 Traffic Snapshot (20 Minutes)</h2><p>From <code>access_20min.log.txt</code>, I counted:</p><p><strong>~13,400 total requests</strong></p><p>Many from <strong>bots</strong>: Googlebot, Bingbot, IAS</p><p>Many hits on <strong>SSR-heavy pages</strong>: <code>/swimmer/</code>, <code>/rankings/</code>, <code>/records/</code></p><p>That’s roughly:</p><p><strong>670 requests per minute</strong></p><p><strong>11 requests per second</strong></p><h2>📈 What a 2-Core VPS Can Usually Handle</h2><p>Under ideal conditions:</p><p>~<strong>20–50 requests/sec</strong> (API or SSR)</p><p><strong>Assumes</strong> good caching, minimal disk I/O, responsive DB</p><p>But SwimStandards also:</p><p>Uses <strong>Next.js SSR + Feathers.js API</strong></p><p>Runs <strong>MongoDB aggregation</strong> per request</p><p>Handles bot load 24/7</p><p>Shows <strong>socket hang up</strong> errors and <strong>TIME_WAIT</strong> spikes</p><h2>🧠 What That Means</h2><p>You're operating at the <strong>edge of stability</strong> for a 2-core server.</p><p>Red flags:</p><p><code>socket hang up</code> during peak periods</p><p><code>TIME_WAIT</code> sometimes exceeds <strong>300</strong></p><p>MongoDB slow logs show query latency</p><p>High I/O due to bot hits on uncached pages</p><h2>✅ Recommendations</h2><h3>1. Upgrade to a 4-core VPS if:</h3><p>You want margin for real traffic surges</p><p>You want better SSR/API responsiveness</p><p>You want fewer 502/socket errors without blocking bots</p><h3>2. Not upgrading yet? Then:</h3><p>Move MongoDB to its own box (if not already)</p><p>Use <code>pm2 scale feathers-dna 2</code> to add more backend workers</p><p>Enable Redis or file-based caching</p><p>Use Cloudflare or NGINX with caching to protect SSR routes</p><h2>🧪 Commands I Used for Analysis</h2><h3>⏱️ View last 20 mins of top IP traffic</h3><pre><code class="language-plaintext">sudo awk -v d="$(date -d '20 minutes ago' '+%d/%b/%Y:%H:%M')" '$0 ~ d,0' /var/log/apache2/other_vhosts_access.log | awk '{print $2}' | sort | uniq -c | sort -nr | head -20</code></pre><h3>🗂️ Count Requests in Log</h3><pre><code class="language-plaintext">wc -l access_20min.log.txt</code></pre><h3>📊 Check Active Connections (443 port)</h3><pre><code class="language-plaintext">sudo netstat -anp | grep :443 | awk '{print $6}' | sort | uniq -c</code></pre><h3>🔍 Analyze top IPs hitting server (bot check)</h3><pre><code class="language-plaintext">sudo awk '{print $2}' /var/log/apache2/other_vhosts_access.log | sort | uniq -c | sort -nr | head -20</code></pre><h3>🔐 Watch Feathers &lt;-&gt; Next.js traffic (port 5052 example)</h3><pre><code class="language-plaintext">watch -n 1 "netstat -anp | grep :5052 | wc -l"</code></pre><h3>🔗 View connection state summary</h3><pre><code class="language-plaintext">sudo netstat -anp | grep :443 | awk '{print $6}' | sort | uniq -c</code></pre><h3>🧵 Check memory, swap, and load</h3><pre><code class="language-plaintext">free -m
uptime</code></pre><h3>🐢 Slow MongoDB queries (on DB server)</h3><pre><code class="language-plaintext">sudo grep -i "slow query" /var/log/mongodb/mongod.log | jq -c 'select(.attr.durationMillis &gt; 800)' &gt; slow.log</code></pre><h2>🧭 Final Thoughts</h2><p>If your site handles real-time rankings and dynamic pages like mine, a 2-core box can survive — but it won’t thrive. The more traffic (bot or not), the harder it gets to avoid dropped connections, long delays, and random SSR failures.</p><p>For me, this test showed it’s time to move up — or keep scraping by with every tweak possible.</p>

On May 28, I reviewed 20 minutes of live traffic logs from my website to evaluate whether my 2-core server could keep up with demand. The data showed over 13,000 requests — roughly 11 per second — with a mix of bot and real user traffic hitting SSR pages and API routes. While the server handled it, signs of strain appeared: spikes in TIME_WAIT, socket hang-up errors, and MongoDB lag. The conclusion? My setup is nearing its safe limits. Upgrading to 4 cores or offloading some processing (e.g., MongoDB or caching) may be necessary to maintain performance and stability as traffic grows.

Is My 2-Core Server Enough? Real Traffic Test & Recommendations

<p>When deploying Node.js applications with PM2, it’s important to understand how cluster mode, instances, and CPU cores interact — especially when you’re combining multiple apps like a Next.js frontend and a Feathers.js backend.</p><p>This post breaks down the core concepts and lessons learned from setting up <code>next-dna</code> and <code>feathers-dna</code> with PM2 cluster mode.</p><figure class="image"><img src="https://images.unsplash.com/photo-1635809337847-cc5e79a3969c?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHwzNHx8Y2x1c3RlcnxlbnwwfHx8fDE3NDg0NjY2Nzl8MA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@acton_crawford?utm_source=DNX&amp;utm_medium=referral">Acton Crawford</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>🔧 1. Workers, Instances, Processes, and CPU Cores</h3><p><strong>Instance</strong> in PM2 refers to a copy of your app.</p><p><strong>Worker</strong> is the actual process that runs an instance.</p><p><strong>Cluster mode</strong> allows PM2 to spawn multiple Node.js processes behind a single port using the built-in cluster module.</p><p>Ideally, you run one instance per CPU core, so the workload is evenly distributed.</p><p>You can check your available cores with:</p><pre><code class="language-plaintext">nproc
</code></pre><p>If you set:</p><pre><code class="language-plaintext">instances: "max"
</code></pre><p>PM2 will match your number of CPU cores.</p><h3>✅ 2. Cluster Mode with 1 Instance Is Still Valid</h3><p>Even if you only run 1 instance, using cluster mode still works — and may even be better than fork mode.</p><p><strong>Why?</strong></p><p>Cluster mode sets up a master process that manages workers.</p><p>It keeps the option open to scale later without changing architecture.</p><p>PM2 handles restarts and monitoring more cleanly in cluster mode.</p><p>So yes — cluster mode with 1 instance is valid and sometimes preferable.</p><blockquote><p>⚠️ <strong>Important Note:</strong><br><strong>Cluster mode with only 1 instance does </strong><i><strong>not</strong></i><strong> support true zero-downtime reloads.</strong><br>During a reload, that single worker must exit before a new one is started, resulting in a short downtime window (you may briefly see 503 errors).<br>To enable zero-downtime reloads, you need <strong>at least 2 instances.</strong></p></blockquote><h3>⚠️ 3. Multiple Instances on a Single Core</h3><p>You can run multiple instances on a single core — but you shouldn’t.</p><p><strong>What happens:</strong></p><p>The OS time-slices the CPU across all your workers.</p><p>This leads to context switching, where the CPU keeps switching between processes.</p><p>That can introduce latency and reduce throughput.</p><p>It’s not catastrophic in small apps, but for production workloads, you’ll want at most one worker per core.</p><h3>🔥 4. More Instances Than CPU Cores = Stress</h3><p>If you go overboard (e.g., 4 workers on 2 cores):</p><p>You’re not magically getting 4× performance.</p><p>Instead, your app can become slower due to CPU thrashing.</p><p>If your app is also I/O-bound (e.g., hitting a DB), performance may degrade even further.</p><p>📌 <strong>Tip:</strong> Use <code>instances: os.cpus().length</code> or simply <code>instances: "max"</code> for optimal performance.</p><h3>🛠 5. Feathers.js + PM2 Cluster Needs Manual Tweak</h3><p>By default, if you naively start Feathers.js in cluster mode using:</p><pre><code class="language-plaintext">app.listen(port);
</code></pre><p>Each worker will try to bind to the same port — and crash with:</p><pre><code class="language-plaintext">EADDRINUSE: address already in use
</code></pre><p>You need to tweak your startup logic:</p><pre><code class="language-plaintext">const startServer = async () =&gt; {
  await app.setup();

  if (process.env.NODE_APP_INSTANCE === undefined || process.env.NODE_APP_INSTANCE === '0') {
    await app.listen(port);
    logger.info(`Feathers app listening on port ${port}`);
  } else {
    logger.info(`Worker ${process.env.NODE_APP_INSTANCE} initialized`);
  }
};

startServer();
</code></pre><p>This ensures only one worker binds to the port, and others are still initialized properly to handle requests routed by PM2.</p><h3>🧩 6. Matching Next.js and Feathers.js Instances Improves Stability</h3><p>If your frontend (Next.js) is running 2 instances, and your backend (Feathers.js) is only running 1 (e.g., fork mode):</p><p>You may see errors like:</p><pre><code class="language-plaintext">socket hang up
</code></pre><p>Or long response delays, as the single Feathers.js worker becomes a bottleneck.</p><p><strong>Why?</strong></p><p>Next.js routes API calls to Feathers.js.</p><p>If there are more frontend workers than backend, you’ll have idle frontend capacity, but backend saturation.</p><p>💡 Matching instance counts between <code>next-dna</code> and <code>feathers-dna</code> reduces the risk of these issues.</p><h3>✅ Final Setup That Works Well</h3><p>After much testing, the following setup works stably:</p><pre><code class="language-plaintext">// pm2 config
instances: 1,
exec_mode: "cluster",
</code></pre><p>For both:</p><p><code>next-dna</code> (Next.js)</p><p><code>feathers-dna</code> (Feathers.js)</p><p>This avoids port binding issues, avoids overloading CPU, and keeps things simple — while still using cluster mode for consistency and future scalability.</p><blockquote><p>🔁 For zero-downtime reloads, bump to <code>instances: 2</code> on both apps.</p></blockquote><h3>🚀 TL;DR</h3><p>✅ Use cluster mode, even for 1 instance — it’s cleaner.</p><p>🧠 Match instances to CPU cores — don’t oversubscribe.</p><p>⚙️ Feathers.js needs a startup guard for port binding.</p><p>🔁 <strong>1 instance ≠ zero-downtime</strong> — for that, use 2+ instances.</p><p>📶 Match instance counts between frontend and backend for stability.</p>

When deploying Node.js applications with PM2, it’s important to understand how cluster mode, instances, and CPU cores interact — especially when you’re combining multiple apps like a Next.js frontend and a Feathers.js backend.

This post breaks down the core concepts and lessons learned from setting up next-dna and feathers-dna with PM2 cluster mode.

Understanding PM2 Cluster Mode with Feathers.js and Next.js

<blockquote><p>Besides the file-based caching causing high disk I/O, adjusting the <a target="_blank" rel="noopener noreferrer" href="https://deniapps.com/blog/understanding-pm2-cluster-mode-with-feathersjs-and-nextjs">PM2 configuration</a> appeared to be a key factor in resolving the <strong>socket hang-up</strong> issue.</p></blockquote><p>Starting on <strong>May 22</strong>, SwimStandards.com began experiencing major slowness — high server load, “socket hang up” errors in the Next.js logs, and sluggish API responses. The issue initially seemed to coincide with my integration of new ad scripts (specifically a CMP script added to the header), which I suspected might be blocking Next.js SSR and delaying downstream API calls. But even after removing the script, the issue returned — confirming it wasn’t the root cause.</p><p>That kicked off a full-scale meltdown diagnosis and optimization effort. I:</p><ul><li>Tuned Axios timeout settings (from 10s to 30s),</li><li>Blocked abusive bots and IPs,</li><li>Investigated MongoDB slow queries and reviewed indexes,</li><li>Limited $skip usage in paginated queries,</li><li>Hid expensive features like team records,</li><li>Tweaked Apache settings (MaxConnectionsPerChild=1000, KeepAlive, etc.),</li><li>And tested persistent socket settings (keep-alive, agent pooling).</li></ul><figure class="image"><img src="https://images.unsplash.com/photo-1559484642-dab5f9c9653e?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHwxNXx8bWF5fGVufDB8fHx8MTc0ODIzMzkzMHww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@sxtcxtc?utm_source=DNX&amp;utm_medium=referral">Sorin Gheorghita</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p><strong>But the actual root cause?</strong></p><p>I had enabled <strong>meet result caching</strong> on <strong>May 12</strong> — which generated .json files into the .next/.cache folder using aggregation-heavy requests. These files quickly piled up (over 15,000), and because both bots and users accessed them constantly, the intense disk I/O (up to 5MB/s) crippled performance. Even when I trimmed the cache down to 1,000 files, the slowness persisted. The fix? <strong>I completely disabled the meet result cache</strong> — and everything immediately stabilized.</p><p>Along the way, I also fixed a long-standing bug in the FeathersJS setModel logic and improved database hygiene. I now actively monitor <strong>MongoDB slow query logs</strong>, <strong>netstat</strong>, and system resource usage to stay ahead of future regressions.</p><p>Below are all the shell commands and diagnostic tools I used throughout this painful but productive journey.</p><h3>✅ Server Load &amp; Traffic Debugging Checklist</h3><h3><strong>1. 🔍 Check for suspicious or bot activity</strong></h3><p><strong>Look for bots targeting /.env or other known paths</strong></p><pre><code class="language-css">sudo grep '/.env' /var/log/apache2/access.log | awk '{print $1}' | sort | uniq -c | sort -nr | head</code></pre><p><strong>Check top IPs accessing any vhost</strong></p><pre><code class="language-css">sudo awk '{print $2}' /var/log/apache2/other_vhosts_access.log | sort | uniq -c | sort -nr | head -10</code></pre><p><strong>Check top IPs in the last 20 minutes</strong></p><pre><code class="language-css">sudo awk -v d="$(date -d '20 minutes ago' '+%d/%b/%Y:%H:%M')" '$0 ~ d,0' /var/log/apache2/other_vhosts_access.log | awk '{print $2}' | sort | uniq -c | sort -nr | head -20</code></pre><h3>2. 🚫 Block abusive IPs using iptables</h3><p><strong>List all existing rules with line numbers</strong></p><pre><code class="language-css">sudo iptables -L INPUT -n --line-numbers</code></pre><p><strong>Block IP</strong></p><pre><code class="language-css">sudo iptables -I INPUT 1 -s &lt;IP_ADDRESS&gt; -j DROP</code></pre><p><strong>Unblock IP</strong></p><pre><code class="language-css">sudo iptables -D INPUT -s &lt;IP_ADDRESS&gt; -j DROP</code></pre><h3>3. 📊 Monitor PM2 logs and app errors</h3><pre><code class="language-css">pm2 logs        # Real-time logs
pm2 status      # Check memory/CPU per process</code></pre><h3>4. 🐢 Identify slow MongoDB queries (on DB server)</h3><pre><code class="language-css">sudo grep -i "slow query" /var/log/mongodb/mongod.log | jq -c 'select(.attr.durationMillis &gt; 800)' &gt; slow.log</code></pre><h3>5. 🌐 Monitor TCP socket load (Next.js &lt;-&gt; Feathers.js port)</h3><pre><code class="language-css">watch -n 1 "netstat -anp | grep :5052 | wc -l"</code></pre><h3>6. 🔗 Check HTTPS connection states</h3><pre><code class="language-css">sudo netstat -anp | grep :443 | awk '{print $6}' | sort | uniq -c</code></pre><h3>🔸 Healthy values (approximate):</h3><ul><li><strong>ESTABLISHED</strong>: ~40–60</li><li><strong>TIME_WAIT</strong>: &lt; 50 preferred (but up to ~100 is manageable)</li><li><strong>SYN_RECV</strong>: ~200–300 is okay if from bots, but monitor for spikes</li></ul><p>&nbsp;</p>

In late May, my website faced severe performance issues—high server load, 'socket hang up' errors, and slow responses. Initially, I suspected new ad scripts, but removing them didn’t help. After days of debugging—adjusting Axios timeouts, blocking bots, optimizing the database, and disabling heavy features—I found the root cause. This post shares the steps I took

May Meltdown Fixes: Every Command I Ran to Battle Website Slowness

<p>I recently uncovered one of the most elusive bugs I've ever dealt with — and it had been quietly sitting in my codebase for over 3 years.</p><figure class="image"><img src="https://images.unsplash.com/photo-1523151594509-9d2e49774fec?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw3fHxidWd8ZW58MHx8fHwxNzQ4MjI0ODk5fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@davidclode?utm_source=DNX&amp;utm_medium=referral">David Clode</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>🔍 The Symptom</h3><p>Out of nowhere, MongoDB logs were showing slow queries against the wrong collections:</p><pre><code class="language-plaintext">ns: swimdb.strokes2324
query: { ageGroup: "13-14", ... }
</code></pre><p>But that didn’t make sense — this should have used the <code>best_strokes2324</code> collection.</p><h3>💡 The Suspicion</h3><p>My service uses a dynamic <code>setModel()</code> method to switch MongoDB collections based on query parameters. It looked fine:</p><pre><code class="language-plaintext">setModel = (collectionName) =&gt; {
  this.app.get("mongoClient").then((db) =&gt; {
    this.Model = db.collection(collectionName);
  });
};
</code></pre><p>And in the hook:</p><pre><code class="language-plaintext">await context.service.setModel("best_strokes2324");
</code></pre><p>Looks safe, right? 😅</p><h3>🧨 The Bug</h3><p>The <code>setModel()</code> function <strong>wasn’t returning anything</strong> — meaning that the <code>await</code> did <strong>absolutely nothing</strong>.</p><p>Requests were racing ahead before the model was actually updated. As a result:</p><p>Some queries hit the right collection</p><p>Others hit a stale or incorrect one</p><p>And the worst part: it all looked fine <i>most of the time</i></p><h3>🧠 The Fix</h3><p>One word: <code>async</code>.</p><pre><code class="language-plaintext">setModel = async (collectionName) =&gt; {
  const db = await this.app.get("mongoClient");
  this.Model = db.collection(collectionName);
};
</code></pre><p>Or if you still use <code>.then()</code>:</p><pre><code class="language-plaintext">setModel = (collectionName) =&gt;
  this.app.get("mongoClient").then((db) =&gt; {
    this.Model = db.collection(collectionName);
  });
</code></pre><p>Now <code>await setModel(...)</code> actually waits.</p><h3>🚦 What I Learned</h3><p>A missing <code>return</code> inside an async-like method can silently break everything.</p><p>If your service state depends on timing, you <strong>must</strong> understand whether the mutation is blocking.</p><p>MongoDB slow query logs are gold — they helped me trace the issue when nothing else would.</p><h3>🧠 Bonus: Why We Also Call <code>setModel()</code> in the <code>after</code> Hook</h3><p>You might wonder — if we already called <code>await setModel()</code> in the <code>before</code> hook, why call it again in the <code>after</code> hook?</p><p>The answer is subtle but critical:</p><blockquote><p><code>context.service.Model</code> is shared across all requests.<br>If another request comes in and sets a different model before our <code>after</code> hook runs, we could end up running queries against the wrong collection — even if we set it earlier.</p></blockquote><p>Feathers services are singletons. The model (<code>this.Model</code>) isn't scoped per request — it's global. But <code>context.params</code> <strong>is</strong> request-specific. So we store <code>season</code> in <code>context.params._season</code> during the <code>before</code> hook and <strong>re-call </strong><code><strong>setModel()</strong></code><strong> in the </strong><code><strong>after</strong></code><strong> hook</strong>, just before any late queries like <code>.distinct()</code>.</p><p>That one extra line protects us from race conditions:</p><pre><code class="language-plaintext">await context.service.setModel("collection_prefix" + context.params._season);
</code></pre><p>This ensures the model is always correct at the time it’s needed — even if another user is hammering the server with overlapping requests.<br><br>🏁 Result</p><p>After the fix:</p><p>No more inconsistent collections</p><p>Query times dropped dramatically</p><p>Confidence restored</p>

Just discovered a 3-year-old bug in my Feathers.js service: I was calling await setModel(...), but forgot to return a Promise inside setModel. The result? Queries sometimes hit the wrong MongoDB collection due to race conditions. It worked most of the time — until I checked the slow query logs. One missing return caused years of silent issues. Fixed now, and performance is back. Lesson learned.

🐛 3 Years of Hidden Bug: How a Missing return Led to Slow Queries and Data Mismatch

<blockquote><p>Since the root cause turned out to be unrelated, I found that Axios works fine with its default settings—there’s no need for custom HTTPS agents, keep-alive, or timeout configurations. - 5/28/2025</p></blockquote><p>If you're running a high-traffic Node.js or Next.js app with APIs (like Feathers.js) and noticing high numbers of <code>TIME_WAIT</code> or <code>SYN_RECV</code> in <code>netstat</code>, you're not alone.</p><figure class="image"><img src="https://images.unsplash.com/photo-1508920291026-c344bbfca1ab?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHwxfHxzb2NrZXR8ZW58MHx8fHwxNzQ4MjE5NjAzfDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@nevenkrcmarek?utm_source=DNX&amp;utm_medium=referral">Neven Krcmarek</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>At <strong>SwimStandards.com</strong>, we serve thousands of daily users, mostly over HTTPS. Recently, we noticed:</p><pre><code class="language-plaintext">sudo netstat -anp | grep :443 | awk '{print $6}' | sort | uniq -c

     9 CLOSING
    42 ESTABLISHED
     4 FIN_WAIT1
     7 LAST_ACK
     1 LISTEN
   256 SYN_RECV
   127 TIME_WAIT
</code></pre><p>These numbers raised concerns. Was it normal? Was performance at risk?</p><h3>🧠 Understanding the Terms</h3><p><code><strong>TIME_WAIT</strong></code>: Indicates a closed connection that the OS is keeping open for 60s (default) to prevent delayed packets from interfering with future ones.</p><p><code><strong>SYN_RECV</strong></code>: Part of the TCP handshake. A high number may mean your server is getting a lot of new connection attempts — possibly from bots or slow clients.</p><h3>✅ When Is <code>TIME_WAIT</code> a Problem?</h3><figure class="table"><table class="ui table celled"><thead><tr><th>Count</th><th>Typical Cause</th><th>Problem?</th><th>Action</th></tr></thead><tbody><tr><td>&lt; 500</td><td>Normal HTTPS/HTTP1.1 traffic</td><td>❌ No</td><td>None</td></tr><tr><td>500–2000</td><td>Moderate load, short connections</td><td>⚠️ Maybe</td><td>Monitor</td></tr><tr><td>&gt; 3000–5000</td><td>High turnover or scraping bots</td><td>✅ Yes</td><td>Optimize + tune OS</td></tr></tbody></table></figure><h3>🔧 Steps We Took to Fix It</h3><h4>1. <strong>Enabled Keep-Alive in Axios (Next.js client)</strong></h4><pre><code class="language-plaintext">import https from "https";

const httpsAgent = new https.Agent({
  keepAlive: true,
  keepAliveMsecs: 5000,
  timeout: 30000,
  maxSockets: 100,
  maxFreeSockets: 10,
});

const client = axios.create({
  httpsAgent,
  timeout: 30000,
});
</code></pre><p>This helped reduce the number of new TCP connections created per request.</p><h4>2. <strong>Verified Feathers.js Still Performs Well Without Custom Agents</strong></h4><p>We didn’t customize the Feathers.js server’s HTTP agent. Node.js by default reuses sockets where possible on the server side. No immediate action was needed.</p><h4>3. <strong>Tuned Kernel Parameters via </strong><code><strong>sysctl</strong></code></h4><pre><code class="language-plaintext"># Enable TCP SYN cookies (protect against SYN flood)
sudo sysctl -w net.ipv4.tcp_syncookies=1

# Allow more concurrent connections in backlog queue
sudo sysctl -w net.core.somaxconn=1024

# Optional: Reduce TIME_WAIT retention
sudo sysctl -w net.ipv4.tcp_fin_timeout=30
sudo sysctl -w net.ipv4.tcp_tw_reuse=1
</code></pre><p>To persist these changes:</p><pre><code class="language-plaintext">sudo nano /etc/sysctl.conf
# Add or uncomment:
net.ipv4.tcp_syncookies=1
net.core.somaxconn=1024
net.ipv4.tcp_fin_timeout=30
net.ipv4.tcp_tw_reuse=1
</code></pre><h3>🧪 How to Monitor</h3><p>Check connection states with:</p><pre><code class="language-plaintext">sudo netstat -anp | grep :443 | awk '{print $6}' | sort | uniq -c
</code></pre><p>Check which IPs are flooding your server:</p><pre><code class="language-plaintext">ss -tn state syn-recv | awk 'NR &gt; 1 {split($5,a,":"); print a[1]}' | sort | uniq -c | sort -nr | head
</code></pre><h3>Observation</h3><p>After these changes, we monitored TCP states with:</p><pre><code class="language-css">sudo netstat -anp | grep :443 | awk '{print $6}' | sort | uniq -c</code></pre><p>While the number of <code>SYN_RECV</code> and <code>TIME_WAIT</code> states <i>did</i> fluctuate, we noticed an increase in <code>TIME_WAIT</code> in some cases — suggesting that <code>somaxconn=1024</code> may not be ideal for our traffic.</p><h3>Reverting <code>somaxconn</code></h3><p>We reverted to the Linux default:</p><pre><code class="language-css">sudo sysctl -w net.core.somaxconn=128</code></pre><p>And updated <code>/etc/sysctl.conf</code>:</p><pre><code class="language-css"># net.core.somaxconn = 1024</code></pre><p>Then reloaded again:</p><pre><code class="language-css">sudo sysctl -p</code></pre><p>This helped stabilize things, and we learned that tuning these values isn’t always linear — it depends on how your app handles slow handshakes, timeouts, and bot traffic.</p><h2>Conclusion</h2><ul><li><strong>Keep-alive helps</strong> reduce new socket overhead.</li><li><strong>SYN cookies</strong> protect against SYN floods.</li><li><strong>Higher somaxconn</strong> isn't always better — monitor TCP state.</li></ul><h3>🟢 Final Thoughts</h3><p><code>TIME_WAIT</code> is normal — only take action if it causes port exhaustion.</p><p><code>SYN_RECV</code> spikes are often bot-related and may be harmless if your server is handling them.</p><p>Keep-alive agents in Axios reduce load and improve socket reuse.</p><p>Use <code>sysctl</code> carefully — tune only when your server shows real-world pressure.</p><p>If you're running an API + Next.js combo like us and want to reduce connection churn and socket errors, these steps are low-risk and effective.</p>

To reduce high SYN_RECV and TIME_WAIT states on our Next.js + Feathers.js server, we enabled HTTP keep-alive in Axios, monitored socket usage, and adjusted system settings (tcp_syncookies=1, somaxconn=1024). While keep-alive helped reduce overhead, increasing somaxconn didn’t clearly improve results and may need to be reverted. Final tuning may also depend on upstream traffic and reverse proxy behavior.

Managing High TIME_WAIT and SYN_RECV in Production: Lessons from SwimStandards.com

<p>When working with MongoDB queries, it's common to use <code>$exists: false</code> to filter out documents missing a certain field. But did you know this pattern can dramatically affect performance — depending on the context?</p><figure class="image"><img src="https://images.unsplash.com/photo-1656908140565-f1947922002f?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw3fHxudWxsfGVufDB8fHx8MTc0ODE0MDMzNnww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@pushparaj_albert?utm_source=DNX&amp;utm_medium=referral">Pushparaj S</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Let’s explore why, and when you should consider switching to <code>{ field: null }</code> instead.</p><h3>🔍 The Problem with <code>$exists: false</code></h3><p>Consider a collection with 1M swim records and a query like this:</p><pre><code class="language-plaintext">db.records.find({
  event: "50FR",
  type: "LCM",
  ageGroup: "13-14",
  isWeak: { $exists: false }
})
</code></pre><p>This looks harmless. But if <code>isWeak</code> is not indexed, MongoDB must:</p><p>Use the index to filter <code>event</code>, <code>type</code>, <code>ageGroup</code></p><p>Then <strong>fetch every matching document</strong> to check if <code>isWeak</code> is missing</p><p>This fetch is expensive — especially under load or on cold cache. It turns a <strong>fast index scan</strong> into a <strong>slow document scan</strong>.</p><h3>✅ The Better Alternative: <code>{ field: null }</code></h3><p>You can rewrite the query as:</p><pre><code class="language-plaintext">db.records.find({
  event: "50FR",
  type: "LCM",
  ageGroup: "13-14",
  isWeak: null
})
</code></pre><p>This has two advantages:</p><p>It matches <strong>both missing fields</strong> and <strong>fields explicitly set to </strong><code><strong>null</strong></code></p><p>MongoDB <strong>can use an index</strong> if <code>isWeak</code> is part of it</p><p>That means your query may become <strong>index-covered</strong>, avoiding document fetches entirely.</p><h3>⚠️ When <code>$exists: false</code> <i>is okay</i></h3><p>There’s one major exception: <strong>text search</strong>.</p><p>Consider this search query:</p><pre><code class="language-plaintext">db.swimmers.find({
  $text: { $search: "cai" },
  dupSlug: { $exists: false }
})
</code></pre><p>This performs well <strong>even without indexing </strong><code><strong>dupSlug</strong></code> because:</p><p>MongoDB runs the <code>$text</code> filter first using the text index</p><p>It then filters the small result set using <code>dupSlug</code></p><p>So <code>$exists: false</code> is okay <strong>when it follows a highly selective condition</strong> like text search.</p><h3>💡 Indexing Strategy Tip</h3><p>If you frequently filter by missing fields:</p><p>Consider using <code>{ field: null }</code> in queries</p><p>And <strong>include </strong><code><strong>field</strong></code><strong> in your compound index</strong></p><p>Example:</p><pre><code class="language-plaintext">db.best_strokes.createIndex({
  event: 1,
  type: 1,
  ageGroup: 1,
  sex: 1,
  isWeak: 1
})
</code></pre><p>Then filter with <code>isWeak: null</code> for full index optimization.</p><h3>🧪 TL;DR</h3><figure class="table"><table class="ui table celled"><thead><tr><th>Query Pattern</th><th>Index Friendly?</th><th>Notes</th></tr></thead><tbody><tr><td><code>{ isWeak: { $exists: false } }</code></td><td>❌ No</td><td>Needs full document fetch</td></tr><tr><td><code>{ isWeak: null }</code></td><td>✅ Yes</td><td>Works with compound indexes</td></tr><tr><td><code>$text + $exists</code></td><td>✅ Often</td><td>Safe if <code>$text</code> is selective</td></tr><tr><td>Partial index on <code>$exists: false</code></td><td>❌ Not allowed</td><td>Must use <code>null</code></td></tr></tbody></table></figure><p>Need help tuning your swim data queries? I’ve got scripts, <code>explain</code> outputs, and index plans for all major patterns. Let me know if you want a follow-up post with <code>explain()</code> walkthroughs.</p>

Using { field: null } in MongoDB queries is often more efficient than { field: { $exists: false } }, because it allows the query to leverage indexes and avoid costly document fetches. We improved performance by switching from $exists: false to isWeak: null and updating the index to include isWeak, reducing query time from over 2 seconds to under 40ms. $exists is still safe in text search filters, but for most use cases, null is the better choice for speed and index optimization.

⚡️ MongoDB $exists vs null: Query Optimization Deep Dive

<p>If you’ve recently seen an error like:</p><pre><code class="language-plaintext">431 Request Header Fields Too Large  
Apache/2.4.52 (Ubuntu) Server</code></pre><p>Don’t worry — it’s a known issue, and here’s what it means (plus how to resolve it).</p><figure class="image"><img src="https://images.unsplash.com/photo-1576717585968-8ea8166b89b8?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw1Mnx8Y29va2llfGVufDB8fHx8MTc0Nzc5MjQ5MXww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@norevisions?utm_source=DNX&amp;utm_medium=referral">No Revisions</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>⸻</p><p>🧠 <strong>What is a 431 Error?</strong></p><p>This error occurs when your browser sends too much data in the request headers — usually due to large or excessive cookies stored for the site. Web servers like Apache and Nginx have limits on how much header data they can process. If those limits are exceeded, access is denied, and the 431 error is triggered.</p><p>⸻</p><p>🔍 <strong>What Caused It in My Case?</strong></p><p>In my case, the error was caused by a cookie named <code>acv</code>, which had grown to over <strong>3.4KB</strong> in size. Combined with other tracking and consent cookies (from ad partners and CMPs), the total header size pushed Apache beyond its default threshold — leading to a sitewide access issue for users.</p><p>Worse, once it happened, visitors had to manually clear cookies just to get back in.</p><p>⸻</p><p>🛠️ <strong>How I Fixed It (as a Site Owner)</strong></p><p>I implemented three changes:</p><p><strong>Increased the Apache header limits</strong> by adding this to the server config:</p><pre><code class="language-plaintext">LimitRequestFieldSize 16384
LimitRequestLine 8190</code></pre><p><strong>Created a custom 431 error page</strong> (<code>/431.html</code>) that now explains the issue and offers:</p><p>A “Clear Cookies” button to quickly reset</p><p>A contact link to report issues</p><p><strong>Monitored cookie size and behavior</strong>, especially from 3rd-party ad/CMP scripts, to catch anything unusually large or persistent.</p><p>⸻</p><p>👩‍💻 <strong>If You’re a Visitor and See This Error</strong></p><p>Try clearing cookies for the site</p><p>Refresh the page</p><p>Or use the contact form on the error page to get help</p><p>⸻</p><p>📌 <strong>Final Note</strong></p><p>If you run a site with ad tech integrations or a CMP, it’s a good idea to monitor cookie sizes and server limits. A single cookie update from a vendor can break access for users without warning.</p><p>Adding a custom error page and a way for users to recover quickly goes a long way in preserving trust and usability.</p>

A quick explanation of the “431 Request Header Fields Too Large” error, what causes it (often oversized cookies), how users can fix it by clearing site data, and what we’re doing to prevent it in the future. This guide is especially useful if you encountered the issue on SwimStandards or any site using modern ad consent scripts.

⚠️ “431 Request Header Fields Too Large” Error – What It Is and How to Fix It

<p>If you've ever run into a confusing MongoDB auth error — where your user works in <code>mongosh</code> but not in your app — the culprit may be a subtle but critical detail: the <code><strong>authSource</strong></code>.</p><figure class="image"><img src="https://images.unsplash.com/photo-1472148083604-64f1084980b9?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw4fHxjb25uZWN0aW9ufGVufDB8fHx8MTc0NzI3NjM0N3ww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@heytowner?utm_source=DNX&amp;utm_medium=referral">JOHN TOWNER</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h2>The Problem</h2><p>You create a user like this:</p><pre><code class="language-plaintext">db.createUser({
  user: "nodebb",
  pwd: "dfd!",
  roles: [
    { role: "readWrite", db: "nodebb" },
    { role: "clusterMonitor", db: "admin" }
  ]
})
</code></pre><p>Then try to connect from PHP, Node.js, or another client:</p><pre><code class="language-plaintext">mongodb://nodebb:dfd!@localhost:27017/nodebb?authSource=admin
</code></pre><p>✅ It works in <code>mongosh</code> if you do this:</p><pre><code class="language-plaintext">mongosh -u nodebb -p --authenticationDatabase nodebb
</code></pre><p>❌ But your app fails to connect.</p><p>Why?</p><h2>The Reason: <code>authSource=admin</code> is Wrong (in this Case)</h2><p>When you use <code>authSource=admin</code>, MongoDB looks for the user in the <code><strong>admin</strong></code> database. But in this example, the user was created in the <code><strong>nodebb</strong></code> database.</p><p>Unless you explicitly created the user in <code>admin</code>, you must match the <code>authSource</code> to the <strong>database where the user was created</strong>.</p><h2>The Fix</h2><p>If you created the user in the <code>nodebb</code> database, then use:</p><pre><code class="language-plaintext">mongodb://nodebb:dfd%21@localhost:27017/?authSource=nodebb
</code></pre><blockquote><p>✅ Note: <code>!</code> must be URL encoded as <code>%21</code></p></blockquote><p>Now everything works:</p><p><code>MongoDB\Client-&gt;selectDatabase('nodebb')</code> in PHP</p><p><code>mongoose.connect(...)</code> in Node.js</p><p>Any other MongoDB client</p><h3>Do I need to include <code>/nodebb</code> in the URI path?</h3><p>No — you only need to include <code>authSource=nodebb</code>.<br>The <code>/nodebb</code> path in the URI (i.e. <code>...@localhost:27017/nodebb</code>) simply sets the <strong>default database</strong> used if you don’t call <code>selectDatabase()</code> in your code.</p><p>If you do call <code>selectDatabase('nodebb')</code> manually — which you should — the path segment is optional.</p><h2>TL;DR</h2><p><code>authSource</code> must match the database <strong>where the user was created</strong></p><p>If you omit it, some drivers may default to <code>admin</code>, which fails silently</p><p>Always URL-encode special characters in passwords in your Mongo URI</p><p>The <code>/database</code> path in the URI is optional if you manually call <code>selectDatabase()</code> in code</p><p>This tiny mistake can cause hours of frustration — now you know how to avoid it.</p>

This post explains a common MongoDB authentication mistake: using the wrong authSource in your connection string. It shows why your CLI login may work while your app fails, how to properly set authSource, whether you need the /database path in the URI, and how to avoid hidden auth errors in PHP, Node.js, or any MongoDB client.

MongoDB authSource Gotcha: Why Your User Works on the CLI but Fails in Code

<p>If you're trying to rewrite clean URLs like <code>/results/summer-meet</code> to <code>/results/view.php?slug=summer-meet</code> in Nginx, everything might seem simple... until your <code>.php</code> files start <strong>downloading instead of executing</strong>.</p><p>Here’s what’s going wrong — and how to fix it.</p><figure class="image"><img src="https://images.unsplash.com/photo-1613905780946-26b73b6f6e11?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHwxfHxzdHVwaWR8ZW58MHx8fHwxNzQ3MjczMDY0fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@elimendeinagella?utm_source=DNX&amp;utm_medium=referral">Elimende Inagella</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h2>❌ The Common (Broken) Approach Using Regex</h2><pre><code class="language-plaintext">location ~ ^/(results|standards|meet|heat-sheets|psych-sheets)/ {
    try_files $uri $uri/ @rewrite_view;
}

location @rewrite_view {
    rewrite ^/(results|standards|meet|heat-sheets|psych-sheets)/([a-z0-9\-]+)$ /$1/view.php?slug=$2 last;
}
</code></pre><p><strong>Why it fails:</strong></p><p>The <code>location ~</code> block is a <strong>regex match</strong>.</p><p>Nginx gives regex blocks <strong>higher priority</strong>, and once it matches this block, it <strong>does not fallback</strong> to your global <code>.php</code>handler like:</p><pre><code class="language-plaintext">location ~ \.php$ { ... }
</code></pre><p>Result: <code>/results/view.php</code> gets matched by the regex block, and since there's no <code>.php</code> handler <strong>inside</strong>, Nginx serves it as a static file — or downloads it.</p><h2>✅ Working Alternative (Split Routers, No Regex)</h2><p>The safest way is to <strong>avoid regex</strong> and use one <code>location</code> per folder:</p><pre><code class="language-plaintext">location /results/ {
    try_files $uri $uri/ @rewrite_results;
}

location @rewrite_results {
    rewrite ^/results/([^/]+)$ /results/view.php?slug=$1 last;
    return 404;
}
</code></pre><p>Repeat for each folder (<code>/standards/</code>, <code>/meet/</code>, etc.).</p><p>This works because:</p><p><code>location /results/</code> is a <strong>prefix match</strong>, not regex</p><p>If the file exists (like <code>view.php</code>), it falls through to the global <code>.php$</code> handler correctly</p><p>No more downloads. Clean, clear routing.</p><h2>✅ Advanced Fix: Keep Regex Router + Add Folder-Specific <code>.php</code> Handler</h2><p>If you really want to keep the compact regex style, you <strong>must re-define</strong> <code>.php</code> handling for those folders:</p><pre><code class="language-plaintext">location ~ ^/(results|standards|meet|heat-sheets|psych-sheets)/ {
    try_files $uri $uri/ @rewrite_view;
}

location ~ ^/(results|standards|meet|heat-sheets|psych-sheets)/.*\.php$ {
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/run/php/php8.2-fpm-swimsnap.sock;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
}
</code></pre><p>This ensures PHP files inside those folders (like <code>view.php</code>) still get routed through PHP-FPM correctly.</p><h2>🧠 Final Advice</h2><p>Prefer <strong>prefix-based blocks</strong> (<code>location /results/</code>) unless you have a strong reason for regex</p><p>If you use <code>location ~</code>, you <strong>must handle </strong><code><strong>.php</strong></code> within the same scope or via more specific regex</p><p>Always verify with <code>nginx -t</code> and test <code>.php</code> execution directly to avoid silent errors</p><p>If you're seeing downloads or blank responses, chances are it's not your rewrite — it's Nginx falling into the wrong block.</p>

This post explains why Nginx PHP rewrites using regex-based location ~ blocks often fail — causing .php files to download instead of execute. It walks through a reliable no-regex solution using prefix-based locations and shows how to fix the regex approach by adding folder-specific .php handlers. Ideal for anyone implementing clean URLs in PHP with Nginx.

Why Your Nginx PHP Rewrites Might Fail — and How to Fix Them

<p>When maintaining an open-source PHP library that integrates with MongoDB, one of the early design questions is:</p><blockquote><p><i>"Should I support multiple versions of </i><code><i>mongodb/mongodb</i></code><i>?"</i><br><i>"Can I safely commit a </i><code><i>composer.lock</i></code><i> if I do?"</i></p></blockquote><figure class="image"><img src="https://images.unsplash.com/photo-1599507593362-50fa53ed1b40?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHwyfHxwaHB8ZW58MHx8fHwxNzQ3MjU2NDk5fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@benofthenorth?utm_source=DNX&amp;utm_medium=referral">Ben Griffiths</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>We initially tried to support both version <code>^1.20</code> and <code>^2.0</code> of <code>mongodb/mongodb</code>, with this in our <code>composer.json</code>:</p><pre><code class="language-plaintext">"require": {
  "mongodb/mongodb": "^1.20 || ^2.0"
}
</code></pre><p>At first glance, this seems like a flexible, future-proof approach — until you run into Composer's core behavior:</p><h2>The Hidden Limitation: <code>composer.lock</code> Can Only Lock to One Version</h2><p>Even if your <code>composer.json</code> allows both:</p><pre><code class="language-plaintext">"mongodb/mongodb": "^1.20 || ^2.0"
</code></pre><p>When you run <code>composer update</code>, Composer:</p><p>Detects the current PHP environment and installed extensions (like <code>ext-mongodb</code>)</p><p>Resolves a compatible version</p><p><strong>Locks that version into </strong><code><strong>composer.lock</strong></code></p><p>This means your <code>composer.lock</code> will <strong>always contain just one version</strong> — either <code>1.x</code> or <code>2.x</code>.</p><p>So what happens if someone installs your library in a different environment?</p><p>If they run <code>composer install</code> and their <code>ext-mongodb</code> version doesn’t match the one used to generate the lock file, <strong>Composer throws an error</strong>.</p><p>This defeats the goal of being "compatible with both."</p><h2>Why This Matters for Open Source Libraries</h2><p>Open source libraries should be flexible. You want developers to:</p><pre><code class="language-plaintext">composer require your/library
</code></pre><p>And have Composer resolve the correct version for their environment — whether they’re using <code>ext-mongodb 1.x</code> or <code>2.x</code>.</p><p>But if you commit a <code>composer.lock</code> locked to <code>mongodb/mongodb 2.x</code>, and a user has <code>ext-mongodb 1.x</code>, they'll get:</p><pre><code class="language-plaintext">Your lock file does not contain a compatible set of packages...
</code></pre><p>Even though your <code>composer.json</code> technically supports their setup.</p><h2>Our Solution: Drop 1.x, Move Forward</h2><p>To avoid this ambiguity and keep the setup clean, we decided to:</p><p><strong>Drop support for </strong><code><strong>mongodb/mongodb 1.x</strong></code></p><p>Require:</p><pre><code class="language-plaintext">"mongodb/mongodb": "^2.0",
"ext-mongodb": "^2.0"
</code></pre><p>Commit a consistent <code>composer.lock</code> so all installs are reliable</p><p>This simplifies everything:</p><p>No compatibility forks</p><p>No runtime surprises</p><p>Clear expectations for contributors and users</p><h2>Final Recommendation</h2><p>If you’re maintaining a <strong>library</strong>, and truly want to support multiple extension versions:</p><p>✅ Use <code>^1.20 || ^2.0</code> in <code>composer.json</code></p><p>❌ <strong>Do not commit </strong><code><strong>composer.lock</strong></code></p><p>If you're building an <strong>app</strong> or CLI tool, or you want consistency across environments:</p><p>✅ Commit <code>composer.lock</code></p><p>✅ Stick to a single version of <code>mongodb/mongodb</code> and its required extension</p><h2>TL;DR</h2><p><code>composer.lock</code> only locks <strong>one version</strong>, no matter how flexible your <code>composer.json</code> is.</p><p>Committing the lock file while trying to support multiple versions <strong>breaks cross-environment compatibility</strong>.</p><p>We chose clarity and consistency: <code>mongodb/mongodb ^2.0</code>, and a committed lock file that just works.</p>

This post explains why we dropped support for mongodb/mongodb 1.x in favor of ^2.0, and the challenges of supporting multiple versions in an open-source PHP library. It highlights a key limitation of Composer: even if your composer.json allows multiple versions, composer.lock can only lock one. We walk through the trade-offs, the reasoning behind our decision, and best practices for maintaining compatibility across environments.

Why We Dropped mongodb/mongodb 1.x Support and Committed to ^2.0

<figure class="image"><img src="https://images.unsplash.com/photo-1558946104-35820fc709ba?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw5Nnx8bGF1bmNofGVufDB8fHx8MTc0Njg4NDE4OXww&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@mcovoy?utm_source=DNX&amp;utm_medium=referral">Makoto Tsuka</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><blockquote><p><strong>Turns out the fix was just plugging in the power! ChatGPT’s not quite smart enough to check that yet!</strong></p></blockquote><h3>Problem</h3><p>We noticed that our scheduled <code>cron</code>/<code>launchd</code> job — responsible for restoring backups via <code>scp</code> — was intermittently failing. Logs showed:</p><pre><code class="language-plaintext">SCP failed
</code></pre><p>Upon closer inspection, the failure was <strong>not due to permissions or file corruption</strong>, but rather that <strong>only a partial set of files (e.g., 106 out of 207) were transferred</strong> before the connection dropped.</p><h3>Root Cause</h3><p>The issue was traced back to <strong>macOS going to sleep during the job</strong>. Since the job ran early in the morning with the lid closed, the system entered sleep mid-transfer — interrupting the active <code>scp</code> session.</p><h3>Solution</h3><p>We wrapped the job with <code>caffeinate</code> to prevent sleep during execution:</p><pre><code class="language-xml">&lt;key&gt;ProgramArguments&lt;/key&gt;
&lt;array&gt;
  &lt;string&gt;/usr/bin/caffeinate&lt;/string&gt;
  &lt;string&gt;-s&lt;/string&gt;
  &lt;string&gt;/Users/adamcai/Projects/ss-db/restore.sh&lt;/string&gt;
&lt;/array&gt;
</code></pre><p>The <code>-s</code> flag keeps the <strong>system (not just the display)</strong> awake while the script runs. This ensures that <code>scp</code> completes without being cut off by sleep.</p><h3>Result</h3><p>After applying this fix, the backup restore job ran reliably every morning — with all files successfully transferred.</p>

A scheduled launchd job using scp failed to complete because macOS went to sleep mid-transfer, interrupting the connection. Adding caffeinate -s to the job prevented system sleep during execution, resolving the issue and ensuring all files transferred reliably.

🐞 Cron Job Debug: SCP Failed Due to macOS Sleep Mid-Transfer

<p>When working with Git, merging branches is a fundamental part of collaboration. But not all merges are the same. There are <strong>three main merge strategies</strong> you can use, and each one impacts your repository history differently.</p><figure class="image"><img src="https://images.unsplash.com/photo-1621165755630-367fc1c27da4?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHwxN3x8bWVyZ2V8ZW58MHx8fHwxNzQ2ODQzMTM3fDA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@raimondklavins?utm_source=DNX&amp;utm_medium=referral">Raimond Klavins</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Here’s a clear breakdown of each method:</p><h2>✅ 1. Fast-Forward Merge (when possible)</h2><p>This is the cleanest type of merge. Git simply <strong>moves the pointer</strong> of the current branch forward — no new commit is created.</p><pre><code class="language-plaintext">git merge feature-branch              # if fast-forward is possible
git merge --ff-only feature-branch    # enforce fast-forward only
</code></pre><p><strong>What happens:</strong></p><p>No new commit is created</p><p>Branch history remains linear</p><p>It looks like all work was done directly on <code>main</code></p><p><strong>Use when:</strong> <code>main</code> hasn’t changed since <code>feature-branch</code> was created.</p><h2>🔁 2. Merge Commit (default when branches diverge)</h2><p>This is Git’s default behavior when branches have diverged.</p><pre><code class="language-plaintext">git merge feature-branch
</code></pre><p><strong>What happens:</strong></p><p>Git creates a new <strong>merge commit</strong></p><p>Both branches' histories are preserved</p><p>The history becomes <strong>non-linear</strong>, with a visible merge point</p><p><strong>Use when:</strong> You want to retain the full commit history of both branches.</p><h2>🧱 3. Squash Merge (<code>--squash</code>)</h2><p>This strategy takes all the changes in the feature branch and <strong>squashes them into a single commit</strong>.</p><pre><code class="language-plaintext">git merge --squash feature-branch
git commit -m "Add feature"
</code></pre><p><strong>What happens:</strong></p><p>All commits in the branch are combined into one</p><p>No merge commit is created automatically</p><p>The branch structure is not recorded in history</p><p><strong>Use when:</strong> You want a <strong>clean, one-commit-per-feature</strong> history (e.g., for open-source projects or public repos).</p><h2>⚙️ Other Merge-related Options</h2><figure class="table"><table class="ui table celled"><thead><tr><th>Option</th><th>What it does</th></tr></thead><tbody><tr><td><code>--no-ff</code></td><td>Forces a merge commit even if fast-forward is possible</td></tr><tr><td><code>--ff-only</code></td><td>Fails the merge if fast-forward isn’t possible</td></tr><tr><td><code>--squash</code></td><td>Combines all changes into one new commit</td></tr><tr><td><code>--no-commit</code></td><td>Prepares the merge but doesn’t commit it (review first)</td></tr></tbody></table></figure><h2>🔁 Summary Table</h2><figure class="table"><table class="ui table celled"><thead><tr><th>Strategy</th><th>Command</th><th>New Commit?</th><th>Keeps History?</th><th>Clean?</th><th>Notes</th></tr></thead><tbody><tr><td>Fast-forward</td><td><code>git merge</code> or <code>--ff-only</code></td><td>❌ No</td><td>✅ Yes</td><td>✅ Yes</td><td>Pointer move only</td></tr><tr><td>Merge commit</td><td><code>git merge</code> or <code>--no-ff</code></td><td>✅ Yes</td><td>✅ Yes</td><td>❌ No</td><td>Default if divergent</td></tr><tr><td>Squash</td><td><code>git merge --squash</code></td><td>✅ Yes</td><td>❌ No</td><td>✅ Yes</td><td>Flattens to one commit</td></tr></tbody></table></figure><p>For public projects like <strong>SwimSnap</strong>, adopting <code>--ff-only</code> or squash merges ensures a readable, maintainable Git history. It helps contributors and maintainers stay focused on features, not merge clutter.</p><p>Want help automating your merge strategy or enforcing it on GitHub? Just ask!</p>

When working with Git, merging branches is a fundamental part of collaboration. But not all merges are the same. There are three main merge strategies you can use, and each one impacts your repository history differently.

Git Merge Strategies Explained: Fast-Forward, Merge Commit, and Squash

<p>When working in a Git-based development team, especially in real-world projects, understanding how to manage branches and history is essential. This post explains the practical use of <code>merge</code>, <code>rebase</code>, and <code>squash</code>, as well as how to use <code>demo</code> or <code>staging</code> branches for safer and more flexible workflows.</p><figure class="image"><img src="https://images.unsplash.com/photo-1584786379647-c10852954d2b?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw4fHxtZXJnZXxlbnwwfHx8fDE3NDY4NDMxMzd8MA&amp;ixlib=rb-4.1.0&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@lg17?utm_source=DNX&amp;utm_medium=referral">Lance Grandahl</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h2>Merge, Rebase, and Squash: What's the Difference?</h2><h3>1. <strong>Merge</strong></h3><p>A <code>git merge</code> preserves full commit history by combining branches using a new <strong>merge commit</strong>. This creates a non-linear history with visible branch structure.</p><h4>Example:</h4><pre><code class="language-css">*   Merge branch 'feature/login'
|\
| * Add login UI
| * Hook up auth
* | Update README
|/
* Initial commit
</code></pre><ul><li>✅ Preserves all commits</li><li>✅ Safe for shared branches</li><li>❌ Creates a messier history</li></ul><h3>2. <strong>Rebase</strong></h3><p><code>git rebase</code> moves your feature branch commits on top of the latest <code>main</code>, effectively rewriting history to make it look like your work was created linearly.</p><h4>Example:</h4><pre><code class="language-css">* Add login UI
* Hook up auth
* Update README
* Initial commit
</code></pre><ul><li>✅ Linear, clean history</li><li>✅ Great for solo or short-lived branches</li><li>❌ Rewrites commit hashes — dangerous on shared branches</li><li>✅ Still preserves each individual commit</li></ul><h3>3. <strong>Squash</strong></h3><p>Squashing combines all your commits into a single one during merge (e.g. GitHub's "Squash and merge").</p><h4>Example:</h4><pre><code class="language-css">* Add login feature (squashed)
* Update README
* Initial commit
</code></pre><ul><li>✅ Very clean history (1 commit per feature)</li><li>❌ Original commits are lost</li><li>✅ Ideal for small PRs or open source contributions</li></ul><h2>Resolving Conflicts: Merge vs Rebase</h2><p>When <code>main</code> has moved ahead and your feature branch is behind, you must resolve conflicts <i>before</i> merging.</p><h3>🔁 Option 1: Merge <code>main</code> into your feature branch</h3><pre><code class="language-css">git checkout feature/login
git merge origin/main
</code></pre><p>✅ Safe, preserves history<br>❌ Will create a merge commit</p><h3>🔁 Option 2: Rebase your feature onto <code>main</code></h3><pre><code class="language-css">git checkout feature/login
git rebase origin/main
</code></pre><p>✅ Cleaner history<br>⚠️ Requires force-push if already pushed</p><blockquote><p>Always resolve conflicts <strong>inside your feature branch</strong>, not on <code>main</code>.</p></blockquote><h2>What If You Resolve a Conflict Incorrectly?</h2><p>Git has no way to know if your conflict resolution is <i>logically wrong</i>. If you accidentally delete part of a previously merged feature during rebase/merge, Git will accept it as valid. That’s why it's important to:</p><ul><li>✅ Carefully review changes (<code>git diff</code> / <code>git log -p</code>)</li><li>✅ Run tests or QA before merging</li><li>✅ Use code reviews to catch errors</li></ul><h2>The Role of a <code>demo</code> or <code>staging</code> Branch</h2><p>Many teams use a branch like <code>demo</code>, <code>staging</code>, or <code>dev</code> as an intermediate testing environment.</p><h3>✅ Why Use a Staging Branch</h3><ul><li>Combine multiple feature branches for QA or demos</li><li>Catch integration issues early</li><li>Test experimental features without affecting <code>main</code></li></ul><h3>🔄 Typical Workflow</h3><pre><code class="language-css">git checkout demo
git merge feature/login
git merge feature/auth
</code></pre><p>If a feature is later rejected, you can simply reset <code>demo</code>:</p><pre><code class="language-css">git checkout demo
git reset --hard origin/main
</code></pre><p>✅ Keeps <code>main</code> clean<br>✅ Allows testing in parallel</p><h3>❌ Don't merge <code>demo</code> back into <code>main</code></h3><ul><li>Doing so could bring in unreviewed or abandoned features</li><li>Always merge reviewed features directly into <code>main</code> from their respective branches</li></ul><h2>Summary</h2><figure class="table"><table class="ui table celled"><thead><tr><th>Strategy</th><th>History</th><th>Keeps Commits?</th><th>Clean?</th><th>Safe on Shared Branch?</th></tr></thead><tbody><tr><td>Merge</td><td>Non-linear</td><td>✅ Yes</td><td>❌</td><td>✅ Yes</td></tr><tr><td>Rebase</td><td>Linear</td><td>✅ Yes</td><td>✅</td><td>⚠️ Risky if shared</td></tr><tr><td>Squash</td><td>Linear</td><td>❌ No</td><td>✅✅</td><td>✅ Yes</td></tr></tbody></table></figure><p>When used wisely, these strategies give you the best of both worlds: clean, readable history and safe, flexible workflows.</p><p>If you're managing features across multiple environments, introducing a <code>demo</code> or <code>staging</code> branch lets you test without polluting your production history.</p><p>Use <code>merge</code> for safety, <code>rebase</code> for clarity, and <code>squash</code> for simplicity. Just be deliberate — and communicate with your team.</p>

When working in a Git-based development team, especially in real-world projects, understanding how to manage branches and history is essential. This post explains the practical use of merge, rebase, and squash, as well as how to use demo or staging branches for safer and more flexible workflows.

Understanding Git Merge Strategies: Merge, Rebase, Squash, and the Role of Staging Branches

<p>Working with interactive data visualizations in React is incredibly powerful, especially when using Chart.js via <code>react-chartjs-2</code>. But it can also be tricky—especially when performance and usability are key. In this post, we’ll break down how to build a performant and maintainable chart component with features like:</p><ul><li>Zooming and panning</li><li>Data point navigation</li><li>Tooltips and highlight syncing</li></ul><p>Let’s walk through how we solved performance issues, structured custom hooks, and handled state transitions cleanly.</p><figure class="image"><img src="https://images.unsplash.com/photo-1591696205602-2f950c417cb9?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHw0fHxjaGFydHxlbnwwfHx8fDE3NDUwMjQ1MzZ8MA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@markuswinkler?utm_source=DNX&amp;utm_medium=referral">Markus Winkler</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h2>✅ Core Challenges</h2><p>When building a zoomable/swipable chart to show a swimmer's progress over time, we needed to:</p><ul><li>Prevent the chart from resetting zoom on every re-render</li><li>Highlight selected data points on click</li><li>Add navigation buttons to move to the previous/next visible point</li><li>Keep performance smooth across updates</li></ul><h2>🧠 Hook: <code>useChartNavigation</code></h2><p>This hook manages internal chart state:</p><pre><code class="language-css">const {
  activeIndex,
  setActiveIndex,
  goToPrev,
  goToNext,
  isPrevDisabled,
  isNextDisabled,
  updateVisibleIndices,
  handleResetZoom,
  handleZoom,
  handleChartClick,
} = useChartNavigation(chartRef, data);</code></pre><h3>Key Implementation Details:</h3><ul><li><code><strong>useCallback</strong></code><strong> for all handlers</strong>: Avoids function re-creation and keeps stable references for useEffect deps.</li><li><code><strong>updateVisibleIndices</strong></code>: Calculates visible data points inside the chart area.</li><li><code><strong>activeIndex</strong></code><strong> &amp; tooltip sync</strong>: When you select a point, the tooltip and highlight sync to it.</li><li><strong>Zooming</strong>: Wrapped in <code>handleZoom()</code> and <code>handleResetZoom()</code> with internal tooltip clearing to avoid jitter.</li></ul><h2>🎨 Component: <code>ProgressChart</code></h2><p>This chart has:</p><ul><li>Line and scatter datasets</li><li>Custom tooltips</li><li>Navigation buttons</li><li>Responsive design for mobile</li></ul><h3>Performance Enhancements:</h3><ul><li><code><strong>chartData</strong></code><strong> and </strong><code><strong>options</strong></code><strong> in </strong><code><strong>useMemo</strong></code>: Prevents recalculation unless dependencies change.</li><li><code><strong>&lt;ChartDisplay&gt;</strong></code><strong> is wrapped in </strong><code><strong>React.memo()</strong></code>: Stops full chart reset when only state like <code>activeIndex</code> changes.</li></ul><pre><code class="language-css">const ChartDisplay = React.memo(function ChartDisplay({ chartRef, chartData, options }) {
  return &lt;Chart ref={chartRef} data={chartData} options={options} /&gt;;
});</code></pre><p>Without <code>React.memo</code>, selecting a point (which updates state) would trigger a re-render and reset zoom.</p><h3>Chart Click Handling</h3><pre><code class="language-css">const handleChartClick = useCallback((event) =&gt; {
  const chart = chartRef.current;
  const elements = chart.getElementsAtEventForMode(event, "nearest", { intersect: false }, false);
  const lineElement = elements.find((el) =&gt; el.datasetIndex === 0);
  if (lineElement) setActiveIndex(lineElement.index);
}, [chartRef]);</code></pre><p>We check <code>datasetIndex === 0</code> to only target the line series and ignore scatter points.</p><h2>🔜 Next Steps: Keyboard Navigation</h2><p>To make chart interaction even better:</p><ul><li>Listen for <code>keydown</code> events</li><li>Use <code>ArrowLeft</code> and <code>ArrowRight</code> to trigger <code>goToPrev()</code> and <code>goToNext()</code></li></ul><p>This can be added via:</p><pre><code class="language-css">useEffect(() =&gt; {
  const handleKey = (e) =&gt; {
    if (e.key === 'ArrowLeft') goToPrev();
    if (e.key === 'ArrowRight') goToNext();
  };
  window.addEventListener('keydown', handleKey);
  return () =&gt; window.removeEventListener('keydown', handleKey);
}, [goToPrev, goToNext]);</code></pre><h2>✅ Final Thoughts</h2><p>This setup is a great real-world example of:</p><ul><li>Structuring reusable logic with hooks</li><li>Using <code>useMemo</code> and <code>useCallback</code> effectively</li><li>Avoiding unnecessary chart re-renders</li><li>Supporting user interaction like zoom + navigation</li></ul><p>We’ve now got a powerful, fast, and user-friendly chart experience.</p><p>Let us know if you want to extend this to handle keyboard input or multiple chart types!</p>

Working with interactive data visualizations in React is incredibly powerful, especially when using Chart.js via react-chartjs-2. But it can also be tricky—especially when performance and usability are key. In this post, we’ll break down how to build a performant and maintainable chart component with features like:

Zooming and panning
Data point navigation
Tooltips and highlight syncing

Let’s walk through how we solved performance issues, structured custom hooks, and handled state transitions cleanly.

Optimizing React Charts with Custom Hooks, useMemo, and Chart.js

<p>If you've ever seen this ESLint warning:</p><blockquote><p>React Hook useEffect has a missing dependency: 'yourFunction'. Either include it or remove the dependency array.</p></blockquote><p>You're not alone — and you're probably asking:</p><ul><li>Why is this happening?</li><li>Why does it behave differently in a component vs. a custom hook?</li><li>And do I really need to use <code>useCallback()</code> here?</li></ul><p>Let’s break this down clearly.</p><figure class="image"><img src="https://images.unsplash.com/photo-1652471329872-0f832dcf0769?crop=entropy&amp;cs=tinysrgb&amp;fit=max&amp;fm=jpg&amp;ixid=M3wxNTQwMTB8MHwxfHNlYXJjaHwxMDh8fGhvb2t8ZW58MHx8fHwxNzQ0OTQxOTc4fDA&amp;ixlib=rb-4.0.3&amp;q=80&amp;w=1080"><figcaption>Photo by <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/@ws_coda?utm_source=DNX&amp;utm_medium=referral">W.S. Coda</a> on <a target="_blank" rel="noopener noreferrer" href="https://unsplash.com/?utm_source=DNX&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>✅ useEffect with an empty dependency array</h3><p>This is a very common pattern:</p><pre><code class="language-css">useEffect(() =&gt; {
  updateSomething();
}, []); // Run once after mount</code></pre><p>This works just fine in React — it's saying:</p><blockquote><p>"Run this effect only once after the initial render."</p></blockquote><p>But ESLint (specifically the <code>react-hooks/exhaustive-deps</code> rule) might still complain:</p><blockquote><p>"You’re using <code>updateSomething</code> inside the effect, but it’s not in the dependency array."</p></blockquote><h3>🤔 Why the warning?</h3><p>ESLint is trying to protect you from <strong>accidentally using stale or unstable references</strong> inside effects. It's checking for all <strong>variables and functions used inside </strong><code><strong>useEffect</strong></code>, and if any are omitted from the dependency array, it throws a warning.</p><p>It doesn't care whether your effect is supposed to run once — it just checks that all references are declared.</p><h3>🧩 Why does it behave differently in a component vs a custom hook?</h3><h4>✅ Inside a component:</h4><pre><code class="language-css">function MyComponent() {
  const updateSomething = () =&gt; {
    // do something
  };

  useEffect(() =&gt; {
    updateSomething();
  }, []); // no ESLint warning in many cases

  return &lt;div /&gt;;
}</code></pre><p>ESLint can often see that:</p><ul><li>The function is defined <strong>within the same component scope</strong></li><li>It doesn't depend on props, state, or other reactive values</li><li>It's unlikely to change between renders</li></ul><p>So it might <strong>not warn you</strong> here — but that’s a best-effort guess.</p><h4>⚠️ Inside a custom hook:</h4><pre><code class="language-css">export function useCustomHook() {
  const updateSomething = () =&gt; {
    // logic here
  };

  useEffect(() =&gt; {
    updateSomething();
  }, []); // ESLint WILL complain
}</code></pre><p>In a hook, ESLint has no way of knowing if the function is truly stable.<br>It assumes the function might capture dynamic state or props.</p><p>So it warns you to either:</p><ul><li>Include the function in the dependency array</li><li>Or wrap it in <code>useCallback()</code></li></ul><blockquote><p>✅ <strong>Note:</strong> The name <code>useCustomHook</code> is also critical. React and ESLint enforce hook rules only for functions <strong>starting with </strong><code><strong>use</strong></code>.<br>If you named it <code>customHookFunction</code>, the hook rules wouldn't apply — but you'd break React's expectations for custom hooks.</p></blockquote><h3>✅ How to fix the warning</h3><h4>Option 1: Use <code>useCallback</code></h4><p>Wrap the function in <code>useCallback</code>:</p><pre><code class="language-css">const updateSomething = useCallback(() =&gt; {
  ...
}, []);

useEffect(() =&gt; {
  updateSomething();
}, [updateSomething]);</code></pre><p>Now ESLint sees the function has a stable identity and a clear dependency list.</p><h4>Option 2: Suppress the warning</h4><p>If you're sure the function is stable and doesn't rely on changing variables, you can just suppress the warning:</p><pre><code class="language-css">// eslint-disable-next-line react-hooks/exhaustive-deps
useEffect(() =&gt; {
  updateSomething();
}, []);</code></pre><p>This is perfectly safe when you know what you're doing — especially for things like DOM access or chart updates that only run once.</p><h3>🧠 Why hook names and component names matter</h3><p>React and ESLint apply different rules based on naming:</p><figure class="table"><table class="ui table celled"><thead><tr><th>Function Type</th><th>Naming Convention</th><th>React Behavior</th><th>ESLint Behavior</th></tr></thead><tbody><tr><td>Component</td><td>Capitalized (e.g. <code>MyComponent</code>)</td><td>Treated as component</td><td>Light hook enforcement</td></tr><tr><td>Custom Hook</td><td>Starts with <code>use</code> (e.g. <code>useCustom</code>)</td><td>Treated as hook</td><td>Strict hook rules applied</td></tr><tr><td>Regular Function</td><td>Anything else</td><td>Not treated specially</td><td>Hook rules not applied</td></tr></tbody></table></figure><blockquote><p>✅ Custom hooks <strong>must start with </strong><code><strong>use</strong></code> for React and ESLint to recognize them and apply hook rules properly. Otherwise, calling <code>useEffect</code> or <code>useState</code> inside them can throw runtime or linting errors.</p></blockquote><h3>✅ TL;DR</h3><figure class="table"><table class="ui table celled"><thead><tr><th>Situation</th><th>Should you use <code>useCallback</code>?</th><th>Will ESLint complain?</th></tr></thead><tbody><tr><td>Function used in useEffect</td><td>✅ Yes, recommended</td><td>✅ Probably</td></tr><tr><td>Defined inside a component</td><td>🤷 Optional</td><td>❌ Might not</td></tr><tr><td>Defined inside a custom hook</td><td>✅ Yes</td><td>✅ Will warn</td></tr><tr><td>Using function once on mount</td><td>❌ Not required</td><td>⚠️ Add comment if needed</td></tr></tbody></table></figure><h3>🔚 Final thoughts</h3><ul><li><strong>React is fine</strong> with you calling a function inside <code>useEffect(() =&gt; {}, [])</code>.</li><li><strong>ESLint just doesn't know</strong> whether it's safe — so it errs on the side of caution.</li><li>You can either use <code>useCallback</code> to be explicit, or suppress the warning if you’re confident it’s safe.</li></ul><p>React gives you flexibility — ESLint just gives you suggestions. Use both wisely. ✅</p>

If you've ever seen this ESLint warning:

React Hook useEffect has a missing dependency: 'yourFunction'. Either include it or remove the dependency array.

You're not alone — and you're probably asking:

Why is this happening?
Why does it behave differently in a component vs. a custom hook?
And do I really need to use useCallback() here?

Let’s break this down clearly.

All Posts