The current Deep Learning (DL) landscape is fast-paced and is rife with non-uniform models, hardware/software (HW/SW) stacks, but lacks a DL benchmarking platform to facilitate evaluation and comparison of DL innovations, be it models, frameworks, libraries, or hardware. Due to the lack of a benchmarking platform, the current practice of evaluating the benefits of proposed DL innovations is both arduous and error-prone — stifling the adoption of the innovations.
MLModelScope is a framework- and hardware-agnostic distributed platform for benchmarking and profiling DL models across datasets/frameworks/systems. MLModelScope offers a unified and holistic way to evaluate and inspect DL models, making it easier to reproduce, compare, and analyze accuracy or performance claims of models or systems.
More specifically, MLModelScope:
Note that MLModelScope and CarML are used interchangeably within these documents. CarML (Cognitive Artifacts for Machine Learning) is the internal code name for MLModelScope.
Feel free to dive in! Open an issue or submit PRs. MLModelScope follows the Contributor Covenant Code of Conduct.