redis-hack-urlshortner

Redis Hack Urlshortner

URL Shortener app built for redis x dev hackathon

#docker #expressjs #redis #svelte #sveltekit
svelte logo

Want a Svelte site built?

Hire a Svelte developer
Demo Download

URL Shortener with Node.JS and Redis

A URL shortener application build with ExpressJS, Redis and SvelteKit. Shorten URLs and also get link visit history.

Live Demo

Table of Contents

Screenshots

Overview video

Here's a short video that explains the project and how it uses Redis:

How it works

How the data is stored:

The project uses redis json to store Url records. Each record has following properties

  • originalUrl: string
  • shortName: string
  • createdAt: date
  • updatedAt: date

To easily work with redis project uses Redis OM for Node.js.

  1. Schema definition

    // server/app/om/url.js
import { Entity, Schema } from 'redis-om'
import client from './client.js'

class Url extends Entity { }

const urlSchema = new Schema(Url, {
    originalUrl: { type: 'string' },
    shortName: { type: 'string' },
    createdAt: { type: 'date' },
    updatedAt: { type: 'date' },
})

export const urlRepository = client.fetchRepository(urlSchema)

await urlRepository.createIndex()

  2. Add new Url record

    When app receives a request to create new record, it validates the request and then creates the record using urlRepository as the following

    // server/app/routers/url-router.js
router.post('/urls/', async (req, res) => {
    // validate request confirm the shortName is available
    // also generate short if not provided
    ...
    const url = await urlRepository.createAndSave({
        originalUrl: originalUrl,
        shortName: shortName,
        createdAt: new Date(),
        updatedAt: new Date()
    })
    ...
}

    The project also uses RedisTimeSeries to store the usage data for the Urls. Three different time series are created for each Url record.

    // server/app/routers/url-router.js

router.post('/urls/', async (req, res) => {
    ...
    // Timeseries to store data every time a URL is hit, that is retained
    // for 28 hours and is used to generate hourly usage chart for Url
    await client.ts.create(url.entityId, {
        RETENTION: 28 * 60 * 60 * 1000,
    })

    // Timeseries to aggregate the above data calculating the sum of all
    // the hits every hour and is retained for 33 days.
    // It is used to display the daily usage chart for Url
    await client.ts.create(url.entityId+":hourly", {
        RETENTION: 33 * 24 * 60 * 60 * 1000,
    })

    // Rule to aggregate hourly data and store it 
    await client.ts.createRule(url.entityId, url.entityId+":hourly", "SUM", 3600000)

    // Time series to incrementally keep the record of total hits, retained
    // indefinitely
    await client.ts.create(url.entityId+":hits")
    ...
}

    And every time a shortcut is used, these metrics gets collected as the following

    // server/app/routers/url-router.js -> redirect
...
// Realtime hits data
await client.ts.add(url.entityId, new Date().getTime(), 1)

// Total hits always incremented
await client.ts.incrBy(url.entityId + ":hits", 1)
...

How the data is accessed:

  1. Redirect to Url when short name is provided

    We search the Url repository using the given short name and if the record is found and exists, we use the originalLink present in the record to redirect

    // server/app/routers/url-router.js

const redirect = async (req, res) => {
    // search by shortName and return the first match
    const url = await urlRepository.search().where('shortName').eq(req.params.shortName).return.first();
    
    // throw 404 if record is not found
    if(!url) {
        return res.status(404).send("URL not found");
    }

    // Update Timeseries if record is found
    await client.ts.add(url.entityId, new Date().getTime(), 1)
    await client.ts.incrBy(url.entityId + ":hits", 1)

    // redirect to the originalUrl
    res.redirect(url.originalUrl)
}

  2. Get history of all the short Urls created

    We return all the records using the search on urlRepository.

    // server/app/routers/url-router.js
const getUrls = async (req, res, next) => {
    if(req.params.shortName === "urls") {
        let urls = await urlRepository.search().return.all()
        urls = urls.map((url) => ({
            id: url.entityId,
            shortName: url.shortName,
            originalUrl: url.originalUrl,
            shortUrl: baseUrl + "/" + url.shortName,
            createdAt: url.createdAt,
            updatedAt: url.updatedAt
        }))
        return res.send(urls)
    }
    next()
}

  3. Get hourly/daily metrics of a Url

    The hourly metrics is recrived from the realtime timeseries data, aggregating hourly during the request as the following

    router.get('/urls/:id/usage', async (req, res) => {
    // fetch URL using the repository
    const url = await urlRepository.fetch(req.params.id);

    // throw error if it doesn't exist
    ...
    
    try {
        let from = Math.min(url.createdAt, new Date().getTime() - 24 * 3600000);
        
        // normalize to hour
        from = Math.floor(from / 3600000) * 3600000;
        const to = Math.ceil(new Date() / 3600000) * 3600000;

        const data = await client.ts.range(url.entityId, from, to, {
            AGGREGATION: {
                type: "SUM",
                timeBucket: 60 * 60 * 1000
            }
        })

        // Fily empty dataset with 0s
        const hours = [];
        for(let i = from; i <= to; i += 3600000) {
            hours.push(new Date(i))
        }
        const usage = hours.map((hour) => {
            const hourData = data.find((d) => d.timestamp == hour.getTime())
            return {
                datetime: hour.toString(),
                timestamp: hour.getTime(),
                value: hourData ? hourData.value : 0
            }
        });

        // Also get total hits
        const hits = await client.ts.get(url.entityId+":hits")

        res.send({
            usage: usage,
            hits: hits?.value ?? 0,
            url: {
                id: url.entityId,
                shortName: url.shortName,
                originalUrl: url.originalUrl,
                shortUrl: baseUrl + "/" + url.shortName,
                createdAt: url.createdAt,
                updatedAt: url.updatedAt,
            }
        })
    } catch(e) {
        console.log(e)
        res.status(500).send({error: 'Unexpected error'})
    }
})

    To retrieve the daily usage we use the exact same workflow, however we use the hourly timeseries and aggregate for every day during the request like we aggregate for every hour above.

How to run it locally?

Prerequisites

Local installation

First, clone the repository from github

git clone https://github.com/lohanidamodar/redis-hack-urlshortner

Running the server

  1. Go to redis-hack-urlshortner/server

    # Go to server folder
cd redis-hack-urlshortner/server

# environment variables
cp .env.example .env

# run the server stack
docker compose up -d

  2. View logs to verify the server is running

    docker compose logs app

    And you should see the output similar to the following.

    server-app-1  | Server started at 8000

Running the Client

  1. Go to the client directory and install dependencies

    # Go to client directory
cd redis-hack-urlshortner/client

# install dependencies
npm install

  2. Copy the default .env.example as .env

    cp .env.example .env

  3. Run the dev script to start the development server

    npm run dev

    You should see the output similar to the following.

    VITE v3.0.8  ready in 722 ms

➜  Local:   http://127.0.0.1:5173/
➜  Network: use --host to expose

  4. Visit the provided link in your browser, you should see the home page

Environment Variables

  • Server
    • APP_PORT: port to be used by web server, default is 8000
    • APP_BASE_URL: If you want to provide different base URL for your server that will be used to construct short URL, default is empty. Will automatically use the server's deployment Url if not provided
    • APP_REDIS_URL: Redis connection string in the format redis://..... It is required to successfully connect to redis
    • APP_ENV: Environment of the app, default is production
  • Client
    • APP_SERVER_BASE_URL: Base URL of the server. It is required for client to be able to access the server

Deployment

To make deploys work, you need to create free account on Redis Cloud

Vercel

  • Server

  • Client

Let's get connected:

More Information about Redis Stack

Here some resources to help you quickly get started using Redis Stack. If you still have questions, feel free to ask them in the Redis Discord or on Twitter.

Getting Started

  1. Sign up for a free Redis Cloud account using this link and use the Redis Stack database in the cloud.
  2. Based on the language/framework you want to use, you will find the following client libraries:

The above videos and guides should be enough to get you started in your desired language/framework. From there you can expand and develop your app. Use the resources below to help guide you further:

  1. Developer Hub - The main developer page for Redis, where you can find information on building using Redis with sample projects, guides, and tutorials.
  2. Redis Stack getting started page - Lists all the Redis Stack features. From there you can find relevant docs and tutorials for all the capabilities of Redis Stack.
  3. Redis Rediscover - Provides use-cases for Redis as well as real-world examples and educational material
  4. RedisInsight - Desktop GUI tool - Use this to connect to Redis to visually see the data. It also has a CLI inside it that lets you send Redis CLI commands. It also has a profiler so you can see commands that are run on your Redis instance in real-time
  5. Youtube Videos

Top categories

tailwind daisyui admin template popup mdsvex portfolio blog form ecommerce ui carousel auth dark seo image routing
svelte logo

Want a Svelte site built?

Hire a Svelte developer
Loading Svelte Themes