Introduction

Vielpork is a Rust-powered HTTP downloader designed for performance and extensibility. It offers:

🚀 Multi-threaded downloads for maximum speed
📊 Multiple built-in reporters to adapt to most scenarios
📦 Rich path policy options and template naming support
🔧 Customizable resource resolution strategies for different download scenarios
⏯️ Support for pausing/resuming both global and individual tasks

stateDiagram-v2
    [*] --> GlobalInit
    GlobalInit --> GlobalRunning: start_all()
    GlobalRunning --> GlobalSuspended: pause_all()
    GlobalSuspended --> GlobalRunning: resume_all()
    GlobalRunning --> GlobalStopped: cancel_all()
    GlobalStopped --> [*]
    
    state TaskStates {
        [*] --> TaskPending
        TaskPending --> TaskDownloading: start_task()
        TaskDownloading --> TaskPaused: pause_task()
        TaskPaused --> TaskDownloading: resume_task()
        TaskDownloading --> TaskCanceled: cancel_task()
        TaskDownloading --> TaskCompleted: finish()
        TaskPaused --> TaskCanceled: cancel_task()
        TaskCanceled --> [*]
        TaskCompleted --> [*]
    }
    
    GlobalSuspended --> TaskPaused : propagate
    GlobalStopped --> TaskCanceled : propagate

osynic_downloader: A osu beatmapsets downloader lib & TUI application based on vielpork.

Core Capabilities

Multi-threaded Architecture: Leverage Rust's async runtime for concurrent chunk downloads
Extensible Reporting:
- Built-in reporters: TUI progress bar, CLI broadcast mpsc channel
- Custom reporter implementation via trait
Smart Resolution:
- Custom resolution logic through Resolver trait
Recovery & Resilience:
- Resume interrupted downloads
Progress Tracking:
- Real-time speed calculations
- ETA estimation
- Detailed transfer statistics

Built-in Options

Reporters

TuiReporter: A terminal-based progress bar based on the indicatif library
CliReporterBoardcastMpsc: A reporter that broadcasts progress updates to multiple channels and finalizes them with a single channel ( Usage Example: In Tonic gRPC server streaming, the rx type can only be mpsc, so we need to broadcast the progress to a mpsc channel, then send it to the client through the server)

Resolvers

UrlResolver: A resolver that downloads resources from a URL, just a simple wrapper around reqwest

Custom Components

You can see all traits at vielpork::base::traits and implement your own components.

Custom Reporter

Here are 2 traits that you need to implement with async_trait:
- ProgressReporter: A trait that allows the reporter to handle progress updates
- ResultReporter: A trait that allows the reporter to handle the results of operations or tasks

Custom Resolver

Here is only 1 trait that you need to implement with async_trait:
- ResourceResolver: A trait that allows the resolver to download resources from a specific source

Afterword (or the prologue)

I found the word "viel" and then thought about "rufen", "ekstase", "reichen".

But when I was still hesitating, a good friend came to my dorm and brought me a cup of smoked pork shreds.

So I named it "vielpork", which means a lot of pork shreds.

But in terms of functionality, this downloader is mainly about multi-reporting channel downloads, so it's also a lot of reporting.

"report" is very close to "vielpork", which is also good.

For me, who has been eating free porridge for a week, this name is already very good.

Oh, by the way, spicy boiled pork slices can also be called VielPork. I love it.

Quick Start

The main parts are as follows

Initial example
Flow control
Usage example: OsynicDownloader

After reading the initial example, you can try to run the example code to download the resources you need.

Then, you can try to customize the Reporter and Resolver.

Custom Reporter
Custom Resolver

Initial Example

Installation

Add to your Cargo.toml:

[dependencies]
vielpork = "0.1.0"

Quick Start

use vielpork::downloader::Downloader;
use vielpork::reporters::tui::TuiReporter;
use vielpork::resolvers::url::UrlResolver;
use vielpork::base::structs::DownloadOptions;
use vielpork::base::enums::DownloadResource;
use vielpork::error::Result;

use std::sync::Arc;
use tokio::sync::Mutex;

#[tokio::main]
async fn main() -> Result<()> {
    let options: DownloadOptions = DownloadOptions::default()
        .with_save_path("fetch".to_string())
        .with_concurrency(3);

    let downloader = Downloader::new(options, Box::new(UrlResolver::new()), Box::new(TuiReporter::new()));

    let resources = vec![
        DownloadResource::Url("https://example.com".to_string()),
        DownloadResource::Url("https://example.com".to_string()),
        DownloadResource::Url("https://example.com".to_string()),
        DownloadResource::Url("https://example.com".to_string()),
        DownloadResource::Url("https://example.com".to_string()),
        DownloadResource::Url("https://example.com".to_string()),
        DownloadResource::Url("https://example.com".to_string()),
        DownloadResource::Url("https://example.com".to_string()),
        DownloadResource::Url("https://example.com".to_string()),
    ];

    downloader.start(resources).await?;

    loop {
        tokio::time::sleep(std::time::Duration::from_secs(1)).await;
        // Because of the async nature of the downloader, we need to keep the main thread alive
    }

    Ok(())
}

Flow Control

vielpork provides a complete flow control method to make it easier for you to control the flow when writing asynchronous code.

Global control
- downloader.lock().await.start(resources).await?;
- downloader.lock().await.pause().await?;
- downloader.lock().await.resume().await?;
- downloader.lock().await.cancel().await?;
Task control
- downloader.lock().await.pause_task(resource).await?;
- downloader.lock().await.resume_task(resource).await?;
- downloader.lock().await.cancel_task(resource).await?;

Just use it directly in the code, and vielpork will automatically handle the state transitions of the tasks.

State Transition Diagram

stateDiagram-v2
    [*] --> GlobalInit
    GlobalInit --> GlobalRunning: start_all()
    GlobalRunning --> GlobalSuspended: pause_all()
    GlobalSuspended --> GlobalRunning: resume_all()
    GlobalRunning --> GlobalStopped: cancel_all()
    GlobalStopped --> [*]
    
    state TaskStates {
        [*] --> TaskPending
        TaskPending --> TaskDownloading: start_task()
        TaskDownloading --> TaskPaused: pause_task()
        TaskPaused --> TaskDownloading: resume_task()
        TaskDownloading --> TaskCanceled: cancel_task()
        TaskDownloading --> TaskCompleted: finish()
        TaskPaused --> TaskCanceled: cancel_task()
        TaskCanceled --> [*]
        TaskCompleted --> [*]
    }
    
    GlobalSuspended --> TaskPaused : propagate
    GlobalStopped --> TaskCanceled : propagate

vielpork's state transition is in the form of a finite state machine, with each task having a state, and each task's state transition is finite. This ensures that the task's state transition is reasonable and that unreasonable state transitions do not occur.

Reasonable Global State Transition


#![allow(unused)]
fn main() {
let valid = match (*current, new_state) {
    (DownloaderState::Idle, DownloaderState::Idle) => true,
    (DownloaderState::Idle, DownloaderState::Running) => true,
    (DownloaderState::Running, DownloaderState::Suspended) => true,
    (DownloaderState::Suspended, DownloaderState::Running) => true,
    (DownloaderState::Stopped, DownloaderState::Idle) => true,
    (_, DownloaderState::Stopped) => true,
    _ => false,
};
}

Reasonable Task State Transition


#![allow(unused)]
fn main() {
let valid = match (*current, new_state) {
    (TaskState::Paused, TaskState::Downloading) => true,
    (TaskState::Paused, TaskState::Paused) => true,
    (TaskState::Paused, TaskState::Pending) => true,
    (TaskState::Pending, TaskState::Paused) => true,
    (TaskState::Pending, TaskState::Downloading) => true,
    (TaskState::Downloading, TaskState::Paused) => true,
    (TaskState::Downloading, TaskState::Completed) => true,
    (TaskState::Failed, _) => true,
    (_, TaskState::Failed) => true,
    (_, TaskState::Canceled) => true,
    _ => false,
};
}

Usage Example: OsynicDownloader

osynic_downloader is an efficient osu! beatmap downloader tool based on vielpork, supporting two input formats and parallel downloading, designed for rhythm game players and beatmap managers.

Recommended to use with osynic_serializer to achieve fast serialization of osu! beatmaps.

✨ Features

Dual-mode input: Supports native osu! beatmap set ID list and custom Osynic serialization generated format
Multiple download sources: Currently supports four download sources: OsuDirect, OsuApiV2, SayoApi, and ChimuApi
Concurrency support: Multi-threaded concurrent downloading acceleration (default 4 threads) (please note the concurrency limit of various osu! mirror site APIs! Use it properly!)
Intelligent management: Automatically create directory structure, custom save path
Visual progress: Real-time TUI progress display (supports terminal 256 colors)
Error recovery: State recovery mechanism ensures download integrity

📦 Installation

Precompiled version

cargo install osynic_downloader

Source code compilation

git clone https://github.com/osynicite/osynic_downloader
cd osynic_downloader
cargo build --release

🚀 Quick Start

Basic usage

# Native mode (ID list)
osynic-dl --beatmapsets json/sets.json -o ./osu_maps -c 8
# Osynic mode (song metadata)
osynic-dl --osynic-songs json/songs.json --output ./music

Configuration file example

sets.json (native mode):

{
    "beatmapset_ids": ["114514", "1919810", "1538879"]
}

songs.json (Osynic mode):

[
  {
    "song_id": 1985060,
    "artist_name": "ヒトリエ",
    "mapper_name": "flake",
    "song_name": "日常と地球の額縁 (wowaka x 初音ミク Edit)",
    "no_video": false
  },
    {
    "song_id": 1997071,
    "artist_name": "ナブナ",
    "mapper_name": "Ryuusei Aika",
    "song_name": "始発とカフカ",
    "no_video": false
  }
]

📜 Command Line Options

Option	Short	Default	Description
--beatmapsets	-b	-	Path to native mode JSON file
--osynic-songs	-n	-	Path to Osynic mode JSON file
--source	-s	SayoApi	osu! beatmap download source
--username	-u	-	osu! account (only for OsuDirect/OsuApiV2)
--password	-p	-	osu! password (only for OsuDirect/OsuApiV2)
--output	-o	beatmapsets	Download directory (auto-created)
--concurrency	-c	4	Download concurrency (1-16)
--help	-h	-	Display help information

Supported osu! Download Sources

OsuDirect: Official osu! beatmap download source (osu username and password required)
OsuApiV2: osu!lazer beatmap download source (osu username and password required, Basic authentication)
SayoApi (default): Sayobot beatmap download source (no login required)
ChimuApi: Chimu.moe beatmap download source (no login required)

📌 Notes

Video download adaptation (no_video) is not yet implemented, and related options will be ignored
Download file naming follows the {{filename}} naming rule
Interrupting the download process with Ctrl+C and then rerunning will resume the download
It is recommended to use a stable network connection for the best experience

🤝 Contributing

Contributions are welcome! Please follow these guidelines:

Follow the official Rust coding style
Add test cases for new features
Run cargo fmt and cargo clippy before submitting

📜 License

This project is open-sourced under the MIT License. Please respect the original author's copyright. When using osu! related resources, please follow the osu! community guidelines.

Custom Reporter

In vielpork, Reporter is an interface used to report the progress and results of download tasks to the outside. Reporter consists of two traits, which define a series of methods for reporting the progress and results of download tasks to the outside. The implementation of Reporter can be any type, as long as it implements these two Reporter traits.

The two traits defined in vielport::base::traits are ProgressReporter and ResultReporter. ProgressReporter defines three methods for reporting the start, progress update, and end of download tasks. ResultReporter defines a method for reporting the result of download tasks.


#![allow(unused)]
fn main() {
#[async_trait]
pub trait ProgressReporter
{
    async fn start_task(&self, task_id: u32, total: u64) -> Result<()>;
    async fn update_progress(&self, task_id: u32, progress: &DownloadProgress) -> Result<()>;
    async fn finish_task(&self, task_id: u32, result: DownloadResult) -> Result<()>;
}

#[async_trait]
pub trait ResultReporter
{
    async fn operation_result(&self, operation: OperationType, code: u32, message: String) -> Result<()>;
}

pub trait CombinedReporter: ProgressReporter + ResultReporter + Send + Sync {}
impl<T: ProgressReporter + ResultReporter + Send + Sync> CombinedReporter for T {}

}

Internal Usage Example One: TuiReporter

vielpork provides a built-in TUI reporter TuiReporter for displaying the progress of download tasks in the terminal. TuiReporter implements two traits, ProgressReporter and ResultReporter, to report the progress and results of download tasks to the outside world.

TuiReporter uses the indicatif library to display the progress of download tasks. When a download task starts, TuiReporter creates a new progress bar and updates the status of the progress bar when the download task ends. TuiReporter also reports the result of the download task to the outside world when the download task ends.


#![allow(unused)]

fn main() {
use crate::error::Result;
use crate::base::structs::DownloadProgress;
use crate::base::enums::{DownloadResult, OperationType};
use crate::base::traits::{ProgressReporter, ResultReporter};
use async_trait::async_trait;


#[cfg(feature = "tui")]
use indicatif::{ProgressBar, ProgressStyle,ProgressDrawTarget,MultiProgress};
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::Mutex;

const MAX_CONCURRENT_BARS: usize = 10 ;

// TUI 实现
#[cfg(feature = "tui")]
#[derive(Debug)]
pub struct TuiReporter {
    mp: MultiProgress,
    bars: Arc<Mutex<HashMap<u32, ProgressBar>>>,
}

impl TuiReporter {
    pub fn new() -> Self {
        let mp = Self::setup_global_progress();
        Self {
            mp,
            bars: Arc::new(Mutex::new(HashMap::new())),
        }
    }
    fn setup_global_progress() -> MultiProgress {
        let mp = MultiProgress::new();
        mp.set_draw_target(ProgressDrawTarget::stdout());
        mp
    }

    // 私有方法用于获取或创建进度条
    async fn get_or_create_bar(&self, task_id: u32, total: u64) -> ProgressBar {
        let mut bars = self.bars.lock().await;
        
        if bars.len() >= MAX_CONCURRENT_BARS {
            bars.retain(|_, bar| !bar.is_finished());
        }

        bars.entry(task_id)
            .or_insert_with(|| {
                let bar = ProgressBar::new(total);
                // 关键修改：将进度条添加到 MultiProgress 系统
                let bar = self.mp.add(bar); // 这行是核心修改
                bar.set_style(ProgressStyle::with_template(&format!(
                    "{{spinner:.green}} [{{bar:.cyan/blue}}] {{bytes}}/{{total_bytes}} ({}) {{msg}}",
                    task_id
                ))
                .unwrap_or(ProgressStyle::default_bar())
                .progress_chars("#>-"));
                bar
            });
        
        bars.get(&task_id).unwrap_or(&ProgressBar::hidden()).clone()
    }
}


#[async_trait]
impl ProgressReporter for TuiReporter {
    async fn start_task(&self, task_id: u32, total: u64) -> Result<()> {
        let bar = self.get_or_create_bar(task_id, total).await;
        bar.set_message("Downloading...");
        Ok(())
    }

    async fn update_progress(&self, task_id: u32, progress: &DownloadProgress) -> Result<()> {
        let bar = self.get_or_create_bar(task_id, progress.total_bytes).await;
        bar.set_position(progress.bytes_downloaded);

        let speed = if progress.rate > 1_000_000.0 {
            format!("{:.2} MB/s", progress.rate / 1_000_000.0)
        } else if progress.rate > 1_000.0 {
            format!("{:.2} KB/s", progress.rate / 1_000.0)
        } else {
            format!("{:.0} B/s", progress.rate)
        };

        // 格式化剩余时间
        let eta = if progress.remaining_time.as_secs() > 60 {
            format!("{}m {}s", 
                progress.remaining_time.as_secs() / 60,
                progress.remaining_time.as_secs() % 60)
        } else {
            format!("{}s", progress.remaining_time.as_secs())
        };
        
        bar.set_message(format!("Speed: {} | ETA: {}", speed,eta));

        
        Ok(())
    }

    async fn finish_task(&self, task_id: u32,result: DownloadResult) -> Result<()> {
        let mut bars = self.bars.lock().await;
        if let Some(bar) = bars.remove(&task_id) {
            match result {
                DownloadResult::Success { path ,duration ,.. } => {
                    // 条的颜色变成绿色，还是#>-
                    bar.set_style(ProgressStyle::with_template(&format!(
                        "{{spinner:.green}} [{{bar:.green/blue}}] {{bytes}}/{{total_bytes}} ({}): {{msg}}",
                        task_id
                    ))?.progress_chars("#>-"));
                    let success_message = format!("✅ Done in {}s, saved to {}",duration.as_secs(),path.display());
                    bar.finish_with_message(success_message)
                },
                DownloadResult::Failed { error,.. } => {
                    // 条变成红色
                    bar.set_style(ProgressStyle::default_bar().template(&format!(
                        "{{spinner:.red}} [{{bar:.red/blue}}] {{bytes}}/{{total_bytes}} ({}): {{msg}}",
                        task_id
                    ))?.progress_chars("#>-"));
                    let error_message = format!("❌ Error: {}", error);
                    bar.abandon_with_message(error_message)
                },
                DownloadResult::Canceled => {
                    // 条变成黄色
                    bar.set_style(ProgressStyle::default_bar().template(&format!(
                        "{{spinner:.red}} [{{bar:.yellow/blue}}] {{bytes}}/{{total_bytes}} ({}): {{msg}}",
                        task_id
                    ))?.progress_chars("#>-"));
                    bar.abandon_with_message("⛔ Canceled")
                },
            }
        }
        Ok(())
    }
}

#[async_trait]
impl ResultReporter for TuiReporter {
    async fn operation_result(&self, operation: OperationType, code: u32, message: String) -> Result<()> {
        if code == 200 {
            println!("{}: {}", operation, message);
        } else {
            eprintln!("{}: {}", operation, message);
        }
        Ok(())
    }
}

}

Internal Usage Example Two: CliReporterBoardcastMpsc

vielpork provides a built-in CLI broadcast reporter CliReporterBoardcastMpsc for broadcasting the progress of download tasks to multiple mpsc channels. CliReporterBoardcastMpsc implements two traits, ProgressReporter and ResultReporter, to report the progress and results of download tasks to the outside world.

CliReporterBoardcastMpsc uses tokio::sync::mpsc to broadcast the progress of download tasks. When a download task starts, CliReporterBoardcastMpsc sends progress update messages to multiple mpsc channels, and reports the result of the download task to the outside world when the download task ends.

This reporter's implementation is relatively simple, only need to send messages to multiple mpsc channels in the methods of ProgressReporter and ResultReporter. It was originally used to solve the problem that the rx type in Tonic gRPC server stream can only be mpsc, so we need to broadcast the progress to mpsc channels and then send it to the client through the server.


#![allow(unused)]
fn main() {
use crate::error::Result;
use crate::base::traits::{ProgressReporter, ResultReporter};
use crate::base::structs::DownloadProgress;
use crate::base::enums::{ProgressEvent, DownloadResult, OperationType};
use async_trait::async_trait;

#[derive(Debug,Clone)]
pub struct CliReporterBoardcastMpsc{
    inner_tx: tokio::sync::broadcast::Sender<ProgressEvent>,
    buffer_size: usize,
}
impl CliReporterBoardcastMpsc {
    pub fn new(buffer_size: usize) -> Self {
        let (inner_tx, _) = tokio::sync::broadcast::channel(buffer_size);
        Self { inner_tx, buffer_size }
    }

    pub fn subscribe_mpsc(&self) -> tokio::sync::mpsc::Receiver<ProgressEvent> {
        let (tx, rx) = tokio::sync::mpsc::channel(self.buffer_size);
        let mut inner_rx = self.inner_tx.subscribe();
        
        tokio::spawn(async move {
            loop {
                match inner_rx.recv().await {
                    Ok(event) => {
                        if tx.send(event).await.is_err() {
                            break;
                        }
                    }
                    Err(_) => {
                        tokio::time::sleep(tokio::time::Duration::from_millis(100)).await;
                    }
                }
            }
        });
        
        rx
    }

    // 发送事件的方法
    pub async fn send(&self, event: ProgressEvent) -> Result<usize> {
        self.inner_tx.send(event)?;
        Ok(self.inner_tx.receiver_count())
    }

    // 创建新订阅者
    pub fn subscribe(&self) -> tokio::sync::broadcast::Receiver<ProgressEvent> {
        self.inner_tx.subscribe()
    }
}

#[async_trait]
impl ProgressReporter for CliReporterBoardcastMpsc {
    async fn start_task(&self, task_id: u32, total: u64) ->Result<()> {
        self.send(ProgressEvent::Start { task_id, total }).await?;
        Ok(())
    }

    async fn update_progress(&self, task_id: u32, progress: &DownloadProgress)->Result<()> {
        self.send(ProgressEvent::Update { task_id, progress: progress.clone() }).await?;
        Ok(())
    }

    async fn finish_task(&self, task_id: u32,finish: DownloadResult) ->Result<()>{
        self.send(ProgressEvent::Finish { task_id ,finish}).await?;
        Ok(())
    }
}

#[async_trait]
impl ResultReporter for CliReporterBoardcastMpsc {
    async fn operation_result(&self, operation: OperationType, code: u32, message: String) ->Result<()> {
        self.send(ProgressEvent::OperationResult { operation, code, message }).await?;
        Ok(())
    }
}
}

Custom Resolver

In vielpork, Resolver is a trait used to resolve the source of resources. vielpork provides a built-in UrlResolver for downloading resources from URLs. You can customize the resolution logic by implementing the Resolver trait.

The trait defined in vielport::base::traits is ResourceResolver. ResourceResolver defines a method for resolving the source of resources.


#![allow(unused)]
fn main() {
#[async_trait]
pub trait ResourceResolver: Send + Sync {
    async fn resolve(&self, resource: &DownloadResource) -> Result<ResolvedResource>;
}
}

What Are Resolvers For?

Resolver is used to convert DownloadResource to ResolvedResource. DownloadResource is an enum type that contains different types of resources. ResolvedResource is a struct that contains detailed information required for an HTTP request.


#![allow(unused)]
fn main() {
// The `DownloadResource` enum type defined in `vielpork::base::enums`

#[derive(Debug, Clone, Serialize, Deserialize)]
pub enum DownloadResource {
    Url(String),
    Id(String), 
    Params(Vec<String>),
    HashMap(HashMap<String, String>),
    Resolved(ResolvedResource),
}
}


#![allow(unused)]
fn main() {
// The `ResolvedResource` struct defined in `vielpork::base::structs`
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ResolvedResource {
    pub id: u32,
    pub url: String,
    pub headers: Vec<(String, String)>,
    pub auth: Option<AuthMethod>,
}
}

In this context, AuthMethod supports three authentication methods: Basic, Bearer, andApiKey.


#![allow(unused)]
fn main() {
// The `AuthMethod` enum type defined in `vielpork::base::enums`

#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub enum AuthMethod {
    None,
    Basic { username: String, password: String },
    Bearer { token: String },
    ApiKey { key: String, header: String },
}
}

Internal Usage Example One: UrlResolver

UrlResolver is a simple resolver that downloads resources from a URL.

In fact, UrlResolver is just a simple reqwest wrapper that converts a URL to a ResolvedResource.


#![allow(unused)]

fn main() {
use crate::error::Result;
use crate::base::traits::ResourceResolver;
use crate::base::structs::ResolvedResource;
use crate::base::enums::DownloadResource;
use crate::base::algorithms::generate_task_id;
use async_trait::async_trait;

#[derive(Debug,Clone)]
pub struct UrlResolver {}

impl UrlResolver {
    pub fn new() -> Self {
        Self {}
    }
}

#[async_trait]
impl ResourceResolver for UrlResolver {
    async fn resolve(&self, resource: &DownloadResource) -> Result<ResolvedResource> {
        match resource {
            DownloadResource::Url(url) => {
                Ok(ResolvedResource{
                    id: generate_task_id(url),
                    url: url.clone(),
                    headers: vec![],
                    auth: None,
                })
            }
            DownloadResource::Resolved(resolved) => {
                Ok(resolved.clone())
            }
            _ => {
                Err("Unsupported resource type".into())
            }
        }
    }
}
}

External Usage Example 1: OsuBeatmapResolver

This part can be seen in the source code of osynic_downloader.

osynic_downloader is an osu! beatmap downloader based on vielpork, containing a utility library and a TUI application.

The Resolver it uses is OsuBeatmapResolver, which is a resource resolver that downloads beatmaps from the official osu! website, API, and various mirror sites.

For example:

For Url type resources, return the Url directly;
For Id type resources, generate the Url based on the Id, and use the default download source;
For Params type resources, generate the Url based on the parameters, and use the download source specified by the parameters;
For HashMap type resources, generate the Url based on the HashMap, and use the download source specified by the parameters.


#![allow(unused)]
fn main() {
use vielpork::base::traits::ResourceResolver;
use vielpork::base::structs::ResolvedResource;
use vielpork::base::enums::{DownloadResource,AuthMethod};
use vielpork::base::algorithms::generate_task_id;
use async_trait::async_trait;

use crate::sources::{DownloadSource, DownloadSourceType};
use crate::url::form_url;

#[derive(Debug,Clone)]
pub struct OsuBeatmapsetResolver {}

impl OsuBeatmapsetResolver {
    pub fn new() -> Self {
        Self {}
    }
}

#[async_trait]
impl ResourceResolver for OsuBeatmapsetResolver {
    async fn resolve(&self, resource: &DownloadResource) -> vielpork::error::Result<ResolvedResource> {
        match resource {
            DownloadResource::Url(url) => {
                Ok(ResolvedResource{
                    id: generate_task_id(url),
                    url: url.clone(),
                    headers: vec![],
                    auth: None,
                })
            }
            DownloadResource::Id(id) => {
                let beatmapset_id: u32;
                match id.parse::<u32>() {
                    Ok(id) => {
                        beatmapset_id = id;
                    }
                    Err(_) => {
                        return Err("Invalid beatmapset id".into());
                    }
                }
                let download_source = DownloadSource::from(DownloadSourceType::Default);
                let base_url = download_source.base_url.clone();
                let url = form_url(&base_url, &beatmapset_id, "", "").map_err(|e| e.to_string())?;
                Ok(ResolvedResource{
                    id: beatmapset_id,
                    url: url.clone(),
                    headers: vec![],
                    auth: None,
                })
            }
            DownloadResource::Params(params) => {
                let beatmapset_id: u32;
                let source: String;

                match params.get(0) {
                    Some(id) => {
                        match id.parse::<u32>() {
                            Ok(id) => {
                                beatmapset_id = id;
                            }
                            Err(_) => {
                                return Err("Invalid beatmapset id".into());
                            }
                        }
                    }
                    None => {
                        return Err("Missing beatmapset_id".into());
                    }
                }
                match params.get(1) {
                    Some(src) => {
                        source = src.clone();
                    }
                    None => {
                        return Err("Missing source".into());
                    }
                }
                let download_source = DownloadSource::from(DownloadSourceType::from(source));
                let base_url = download_source.base_url.clone();

                let username: String;
                let password: String;

                let url: String;
                if download_source.requires_osu_credentials {
                    match params.get(2) {
                        Some(name) => {
                            username = name.clone();
                        }
                        None => {
                            return Err("Missing username".into());
                        }
                    }
                    match params.get(3) {
                        Some(pass) => {
                            password = pass.clone();
                        }
                        None => {
                            return Err("Missing password".into());
                        }
                    }
                    if download_source.requires_basic_auth{
                        url = form_url(&base_url, &beatmapset_id, "","").map_err(|e| e.to_string())?;
                        Ok(
                            ResolvedResource{
                                id:beatmapset_id,
                                url: url.clone(),
                                headers: vec![],
                                auth: Some(AuthMethod::Basic { username, password }),
                            }
                        )
                    } else {
                        let hashed_password = format!("{:x}", md5::compute(password));
                        url = form_url(&base_url, &beatmapset_id, &username, &hashed_password).map_err(|e| e.to_string())?;
                        Ok(
                            ResolvedResource{
                                id:beatmapset_id,
                                url: url.clone(),
                                headers: vec![],
                                auth: None,
                            }
                        )
                    }
                } else {
                    url = form_url(&base_url, &beatmapset_id, "", "").map_err(|e| e.to_string())?;
                    Ok(
                        ResolvedResource{
                            id:beatmapset_id,
                            url: url.clone(),
                            headers: vec![],
                            auth: None,
                        }
                    )
                }

            }
            DownloadResource::HashMap(hashmap) => {
                let beatmapset_id: u32;
                let source: String;
                match hashmap.get("beatmapset_id") {
                    Some(id) => {
                        match id.parse::<u32>() {
                            Ok(id) => {
                                beatmapset_id = id;
                            }
                            Err(_) => {
                                return Err("Invalid beatmapset id".into());
                            }
                        }
                    }
                    None => {
                        return Err("Missing beatmapset_id".into());
                    }
                }
                match hashmap.get("source") {
                    Some(src) => {
                        source = src.clone();
                    }
                    None => {
                        return Err("Missing source".into());
                    }
                }
                let download_source = DownloadSource::from(DownloadSourceType::from(source));
                let base_url = download_source.base_url.clone();

                let username: String;
                let password: String;

                let url: String;
                if download_source.requires_osu_credentials {
                    match hashmap.get("username") {
                        Some(name) => {
                            username = name.clone();
                        }
                        None => {
                            return Err("Missing username".into());
                        }
                    }
                    match hashmap.get("password") {
                        Some(pass) => {
                            password = pass.clone();
                        }
                        None => {
                            return Err("Missing password".into());
                        }
                    }
                    if download_source.requires_basic_auth{
                        url = form_url(&base_url, &beatmapset_id, "","").map_err(|e| e.to_string())?;
                        Ok(
                            ResolvedResource{
                                id:beatmapset_id,
                                url: url.clone(),
                                headers: vec![],
                                auth: Some(AuthMethod::Basic { username, password }),
                            }
                        )
                    } else {
                        let hashed_password = format!("{:x}", md5::compute(password));
                        url = form_url(&base_url, &beatmapset_id, &username, &hashed_password).map_err(|e| e.to_string())?;
                        Ok(
                            ResolvedResource{
                                id:beatmapset_id,
                                url: url.clone(),
                                headers: vec![],
                                auth: None,
                            }
                        )
                    }
                } else {
                    url = form_url(&base_url, &beatmapset_id, "", "").map_err(|e| e.to_string())?;
                    Ok(
                        ResolvedResource{
                            id:beatmapset_id,
                            url: url.clone(),
                            headers: vec![],
                            auth: None,
                        }
                    )
                }                
            }
            DownloadResource::Resolved(resolved) => {
                Ok(resolved.clone())
            }
        }
    }
}

}

Change Log

0.1

Extracted the common HTTP downloader logic from the download module of osynic_core, and after supporting custom Reporter and Resolver, made it an independent library vielpork

Roadmap

There are no more plans for the time being, please stay tuned.

Appendix

State Diagrams

stateDiagram-v2
    [*] --> GlobalInit
    GlobalInit --> GlobalRunning: start_all()
    GlobalRunning --> GlobalSuspended: pause_all()
    GlobalSuspended --> GlobalRunning: resume_all()
    GlobalRunning --> GlobalStopped: cancel_all()
    GlobalStopped --> [*]
    
    state TaskStates {
        [*] --> TaskPending
        TaskPending --> TaskDownloading: start_task()
        TaskDownloading --> TaskPaused: pause_task()
        TaskPaused --> TaskDownloading: resume_task()
        TaskDownloading --> TaskCanceled: cancel_task()
        TaskDownloading --> TaskCompleted: finish()
        TaskPaused --> TaskCanceled: cancel_task()
        TaskCanceled --> [*]
        TaskCompleted --> [*]
    }
    
    GlobalSuspended --> TaskPaused : propagate
    GlobalStopped --> TaskCanceled : propagate

Troubleshoting

This library was written in about a morning, so there are definitely many areas that need improvement. At present, it only meets the requirements of my own project and cannot guarantee that it will fully meet everyone's requirements.

So, if there are any problems with the code, or if you have any suggestions, please feel free to submit a PR or Issue, and I will handle it as soon as possible~

If you want to contribute code, please follow these rules:

Follow the official Rust coding style
Include test cases for new features
Run cargo fmt and cargo clippy before submitting