用Rust写的Brotli解压缩器, 可选择地避免使用stdlib

・5 分钟阅读

  • 源代码名称: rust-brotli
  • 源代码网址: https://www.github.com/dropbox/rust-brotli
  • rust-brotli的文档
  • rust-brotli的源代码下载
  • Git URL:
    git://www.github.com/dropbox/rust-brotli.git
  • Git Clone代码到本地:
    git clone https://www.github.com/dropbox/rust-brotli
  • Subversion代码到本地:
    $ svn co --depth empty https://www.github.com/dropbox/rust-brotli
                              Checked out revision 1.
                              $ cd repo
                              $ svn up trunk
              
  • 铁锈 brotli

    crates.ioBuild Status

    项目要求

    将C brotli压缩器的no-stdlib端口直接连接到Rust

    对 Rust stdlib没有依赖关系:

    压缩使用

    Rust brotli目前支持压缩级别是0 - 11,应该与压缩级别0 -9的brotli C压缩引擎相同。建议lg_window_size介于20和22之间。

    io::Read 抽象

    
    
    
    
    let mut input = brotli::CompressorReader::new(&mut io::stdin(), 4096 /* buffer size */,
    
    
     quality as u32, lg_window_size as u32);
    
    
    
    

    然后你可以简单地读取输入,就像任何其他 io::Read 类一样

    io::Write 抽象

    
    
    
    
    let mut writer = brotli::Compressor::new(&mut io::stdout(), 4096 /* buffer size */,
    
    
     quality as u32, lg_window_size as u32);
    
    
    
    

    流复制抽象

    
    
    
    
    match brotli::BrotliCompress(&mut io::stdin(), &mut io::stdout(), quality as u32, lg_window_size as u32) {
    
    
     Ok(_) => {},
    
    
     Err(e) => panic!("Error {:?}", e),
    
    
    }
    
    
    
    

    解压缩用法

    io::Read 抽象

    
    
    
    
    let mut input = brotli::Decompressor::new(&mut io::stdin(), 4096 /* buffer size */);
    
    
    
    

    然后你可以简单地读取输入,就像任何其他 io::Read 类一样

    io::Write 抽象

    
    
    
    
    let mut writer = brotli::DecompressorWriter::new(&mut io::stdout(), 4096 /* buffer size */);
    
    
    
    

    流复制抽象

    
    
    
    
    match brotli::BrotliDecompress(&mut io::stdin(), &mut io::stdout()) {
    
    
     Ok(_) => {},
    
    
     Err(e) => panic!("Error {:?}", e),
    
    
    }
    
    
    
    

    手动内存管理

    有 3个步骤可以在没有stdlib的情况下使用 brotli

    • 设置内存管理器
    • 设置 BrotliState
    • 在循环中,调用 BrotliDecompressStream

    详细

    
    
    
    
    //at global scope declare a MemPool type -- in this case we'll choose the heap to
    
    
    //avoid unsafe code, and avoid restrictions of the stack size
    
    
    
    declare_stack_allocator_struct!(MemPool, heap);
    
    
    
    //at local scope, make a heap allocated buffers to hold uint8's uint32's and huffman codes
    
    
    let mut u8_buffer = define_allocator_memory_pool!(4096, u8, [0; 32 * 1024 * 1024], heap);
    
    
    let mut u32_buffer = define_allocator_memory_pool!(4096, u32, [0; 1024 * 1024], heap);
    
    
    let mut hc_buffer = define_allocator_memory_pool!(4096, HuffmanCode, [0; 4 * 1024 * 1024], heap);
    
    
    let heap_u8_allocator = HeapPrealloc::<u8>::new_allocator(4096, &mut u8_buffer, bzero);
    
    
    let heap_u32_allocator = HeapPrealloc::<u32>::new_allocator(4096, &mut u32_buffer, bzero);
    
    
    let heap_hc_allocator = HeapPrealloc::<HuffmanCode>::new_allocator(4096, &mut hc_buffer, bzero);
    
    
    
    //At this point no more syscalls are going to be needed since everything can come from the allocators.
    
    
    
    //Feel free to activate SECCOMP jailing or other mechanisms to secure your application if you wish.
    
    
    
    //Now it's possible to setup the decompressor state
    
    
    let mut brotli_state = BrotliState::new(heap_u8_allocator, heap_u32_allocator, heap_hc_allocator);
    
    
    
    //at this point the decompressor simply needs an input and output buffer and the ability to track
    
    
    //the available data left in each buffer
    
    
    loop {
    
    
     result = BrotliDecompressStream(&mut available_in, &mut input_offset, &input.slice(),
    
    
     &mut available_out, &mut output_offset, &mut output.slice_mut(),
    
    
     &mut written, &mut brotli_state);
    
    
    
     //just end the decompression if result is BrotliResult::ResultSuccess or BrotliResult::ResultFailure
    
    
    }
    
    
    
    

    这里接口与brotli解压缩器使用的接口相同

    也可以随意使用直接调用Box的自定义分配器。

    讨论
    Fansisi profile image