QueryWeb3 - Web3 Data Analysis Platform

A Go-based backend service for Web3 data analysis platform, providing volume statistics and yield data query functionality.

中文文档

Tech Stack

• Web Framework: Go Fiber
• ORM: GORM
• CLI Framework: Cobra
• Configuration Management: Viper
• Database: MySQL
• Cache System: Redis
• API Documentation: Auto-generated

Project Structure

query-web3/
├── cmd/
│   ├── server/          # Server entry point
│   └── cli/             # CLI tool entry point
├── internal/
│   ├── config/          # Configuration management
│   ├── database/        # Database connection
│   ├── models/          # Data models
│   ├── schemas/         # Request/response structures
│   ├── handlers/        # HTTP handlers
│   ├── server/          # Server configuration
│   ├── cli/             # CLI commands
│   ├── redis/           # Redis cache service
│   ├── cache/           # Cache interface definitions
│   ├── logger/          # Logging system
│   └── health/          # Health checks
├── configs/             # Configuration files
├── docs/                # Documentation directory
├── .env.example         # Environment variables example
└── README.md            # Project documentation

Quick Start

1. Requirements

• Go 1.21+
• MySQL 8.0+

2. Install Dependencies

go mod tidy

3. Configure Database

Create MySQL database:

CREATE DATABASE query_web3 CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

4. Configure Application

Copy environment variables file:

cp .env.example .env

Edit .env file, configure database and Redis connection:

# Database Configuration
QUERY_WEB3_DATABASE_HOST=localhost
QUERY_WEB3_DATABASE_PORT=3306
QUERY_WEB3_DATABASE_USER=root
QUERY_WEB3_DATABASE_PASSWORD=your_password
QUERY_WEB3_DATABASE_NAME=query_web3

# Redis Configuration
QUERY_WEB3_REDIS_HOST=localhost
QUERY_WEB3_REDIS_PORT=6379
QUERY_WEB3_REDIS_PASSWORD=
QUERY_WEB3_REDIS_DB=0
QUERY_WEB3_REDIS_POOL_SIZE=100

5. Run Application

Method 1: Using CLI Tool

# Build CLI tool
go build -o query-web3 cmd/cli/main.go

# Execute database migration
./query-web3 migrate

# Start server
./query-web3 server

Method 2: Direct Server Run

go run cmd/server/main.go

6. Access API

After server starts, access:

• API Root: http://localhost:8000
• API v1: http://localhost:8000/api/v1

API Endpoints

Volume Statistics

POST /api/v1/vol-txns

Request body:

{
  "from_date": "2024-01-01",
  "to_date": "2024-01-31",
  "chain": "ethereum",
  "cycle": "daily"
}

Yield Data Query

POST /api/v1/yield

Request body:

{
  "date": "2024-01-01",
  "chain": "ethereum",
  "asset_type": "defi",
  "return_type": "lending",
  "token": "USDC",
  "page": 1,
  "page_size": 10
}

Metadata Queries

• GET /api/v1/chains - Get blockchain list
• GET /api/v1/cycles - Get statistics cycles list
• GET /api/v1/asset-types - Get asset types list
• GET /api/v1/return-types - Get return types list
• GET /api/v1/tokens - Get tokens list (with chain and asset type filtering)

Development

Build Project

# Build server
go build -o bin/server cmd/server/main.go

# Build CLI tool
go build -o bin/query-web3 cmd/cli/main.go

Run Tests

go test ./...

Code Formatting

go fmt ./...

Deployment

Docker Deployment

FROM golang:1.21-alpine

WORKDIR /app

COPY go.mod go.sum ./
RUN go mod download

COPY . .
RUN go build -o main cmd/server/main.go

EXPOSE 8000

CMD ["./main"]

Environment Variables Configuration

All configurations can be overridden by environment variables:

export QUERY_WEB3_DATABASE_HOST=mysql.example.com
export QUERY_WEB3_DATABASE_PORT=3306
export QUERY_WEB3_DATABASE_USER=app_user
export QUERY_WEB3_DATABASE_PASSWORD=secure_password
export QUERY_WEB3_DATABASE_NAME=query_web3
export QUERY_WEB3_SERVER_PORT=8080

Redis Performance Optimization

QueryWeb3 project integrates Redis cache system, significantly improving system performance and response speed.

Cache Strategy

1. Metadata Cache

• Cache Objects: Blockchain networks, asset types, return types, statistics cycles, token lists
• Expiration Time: 1 hour
• Performance Improvement: 95-99%

2. Query Result Cache

• Cache Objects: Transaction statistics, yield queries, token list query results
• Expiration Time: 30 minutes
• Performance Improvement: 97-98%

Performance Improvement Results

Concurrency Capacity Improvement

• Without Cache: Supports 100-200 concurrent users
• With Cache: Supports 1000-2000 concurrent users
• Concurrency Improvement: 5-10x

Graceful Degradation

• Automatically falls back to database queries when Redis is unavailable
• Logs warnings but doesn't interrupt service
• Ensures high system availability

Detailed Documentation

For more detailed information about Redis performance optimization, please refer to Redis Performance Guide.

License

MIT License