jramapuram/hal

HAL : Hyper Adaptive Learning

Rust based Cross-GPU Machine Learning

HAL : Hyper Adaptive Learning

Rust based Cross-GPU Machine Learning.

Why Rust?

This project is for those that miss strongly typed compiled languages. Rust was chosen specifically because of this Furthermore, we can offer fine grained control of your operations. This means being able to grab dimensions of tensors at any stage, none of that unknown shape nonsense. We can also micro-control steps. An example of this working with each individual forward timesteps on say an LSTM. Usually these are controlled by inner loops [Theano/Tensorflow, etc].

Features

  • Multi GPU [model based] support
  • OpenCL + CUDA + Parallel CPU support
  • LSTM's with internal RTRL [Work in Progress]
  • RNN's [Work in Progress]
  • Perceptrons, AutoEncoders, ConvNets**[TODO]**
  • Optimizers: [SGD, Adam, AdaGrad**[TODO]**]
  • Activations: [Linear, Sigmoid, Tanh, ReLU, LReLU, Softmax]
  • Initializations: [Lecun Uniform, Glorot Normal, Glorot Uniform, Normal, Uniform]
  • Data Gatherers: [SinSource, MNIST**[In Progress], CIFAR10[TODO]**]
  • Loss Functions: [MSE, L2, Cross-Entropy]
  • OpenGL based plotting and image loading, see here for more info
  • Multi GPU [horizontal] support [TODO]

Requirements

  • Rust 1.8 +
  • Download and install ArrayFire binaries based on your operating system.

Use from Crates.io

To use the rust bindings for ArrayFire from crates.io, the following requirements are to be met first.

  1. Download and install ArrayFire binaries based on your operating system.
  2. Set the evironment variable AF_PATH to point to ArrayFire installation root folder.
  3. Make sure you add the path to library files to your path environment variables.
    • On Linux & OSX: do export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$AF_PATH/lib
    • On Windows: Add %AF_PATH%\lib to your PATH environment variable.
  4. Add hal-ml as a dependency in your Cargo.toml

Build from Source

Edit build.conf to modify the build flags. The structure is a simple JSON blob. Currently Rust does not allow key:value pairs to be passed from the CLI. To use an existing ArrayFire installation modify the first three JSON values. You can install ArrayFire using one of the following two ways.

  • Download and install binaries
  • Build and install from source

To build arrayfire submodule available in the rust wrapper, you have to do the following.

This is recommended way to build Rust wrapper since the submodule points to the most compatible version of ArrayFire the Rust wrapper has been tested with. You can find the ArrayFire dependencies below.

  • Linux dependencies
  • OSX dependencies

Examples

Testing

HAL utilizes RUST's test framework to extensively test all of our modules.
We employ gradient checking on individual functions as well as layers.
Testing needs to run on one thread due to our device probing methods.
Furthermore, graphics needs to be disabled for testing [glfw issue].

If you would like to see the results of the test (as well as benchmarks) run:

Credits

  • Thanks to the arrayfire team for working with me to get the rust bindings up.
  • Keras for inspiration as a lot of functions are similar to their implementation (minus the theano nonsense).
  • Dr. Felix Gers for his insight into internal RTRL
  • Dr. Sepp Hochreiter for advise on LSTM's
Issues

Collection of the latest Issues

clive-g-brown

clive-g-brown

Comment Icon5

Only change I made was to Csrgo.toml to update spmc as the version in there is gone. ArrayFire-rust alone compiles fine

error[E0277]: (dyn std::ops::Fn(defines::AfError) + 'static) cannot be shared between threads safely --> arrayfire-rust\src\error.rs:25:1 | 25 | pub static DEFAULT_HANDLE_ERROR: &'static ErrorCallback = &handle_error_general; | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (dyn std::ops::Fn(defines::AfError) + 'static) cannot be shared between threads safely | = help: within &'static (dyn std::ops::Fn(defines::AfError) + 'static), the trait std::marker::Sync is not implemented for (dyn std::ops::Fn(defines::AfError) + 'static) = note: required because it appears within the type &'static (dyn std::ops::Fn(defines::AfError) + 'static) = note: shared static variables must have a type that implements Sync

jramapuram

jramapuram

Comment Icon0

There are quite a few problems using af::array to handle CPU <-> GPU transitions. A swap to ndarray would be cleaner for host data

jramapuram

jramapuram

Comment Icon0

Data loading is slow. To resolve this we need a separate thread(s) that prefetch data into a queue which is used then in the actual model to execute (get_batch_*) calls.

jramapuram

jramapuram

Comment Icon0

Remove the unwrap() calls all over the code and change it to a Result<> type

Information - Updated Apr 04, 2022

Stars: 83
Forks: 10
Issues: 5

Repositories & Extras

utest standard testing for Rust

Make sure your not using this for testing no_std code as it relies on the unstable branch

utest standard testing for Rust

K9 - Rust Testing Library

Snapshot testing + better assertions

K9 - Rust Testing Library

Spec &quot;it&quot; for Rust testing

The test output is like the following

Spec &quot;it&quot; for Rust testing

Shuttle is a library for testing concurrent Rust code

It is an implementation of a number of

Shuttle is a library for testing concurrent Rust code

Let's make a web service and client in Rust

So I'm working on this project Rust regression testing

Let's make a web service and client in Rust

Simple golden file testing for Rust

Add the following to your Cargo manifest

Simple golden file testing for Rust
Http

329

HTTP mocking to test Rust applications

wiremock provides HTTP mocking to perform black-box testing of Rust applications that

HTTP mocking to test Rust applications

just testing rust

cli + advanced cli features

just testing rust
Facebook Instagram Twitter GitHub Dribbble
Privacy