Release notes for SystemDS 3.3.0
SystemDS 3.3 is a minor release. Release 3.3 contains new features and major improvements to existing features.
New Features
Built-in Functions
- Added
sliceLineExtract
for efficient slicing operations.
- Introduced
adasyn
for synthetic data generation in imbalanced datasets.
- Added
GELU Activation Function
(approximation) for deep learning tasks.
- New built-in for
vectorized simple exponential smoothing
.
- Python API Enhancements:
- Added built-ins for
solve
, median
, quantile
, lu
, qr
, cov
, and more.
- Introduced cumulative aggregate functions like
cumsum
, cumprod
, etc.
- Added support for matrix operations like
diag
, trace
, sqrt
, exp
, floor
, and ceil
.
- Optimized matrix multiplication, including multi-threaded dot-product operations.
- Improved performance for CSV and MatrixMarket readers with parallelism.
- Enhanced incremental slice-line pruning for better efficiency.
- Optimized federated operations with improved threading and cost estimation.
Federated Learning
- Fixed threading bugs in federated operations.
- Improved federated planner memoization and added support for weighted covariance.
Compression
- Introduced workload-aware compression for intermediates.
- Enhanced compressed matrix operations with new tests and optimizations.
Scuro Framework
- Added support for multi-head attention layers and LSTM representations.
- Introduced new modalities and data representations for Scuro.
Bug Fixes
- Fixed robustness issues in built-in functions:
- Corrected
trace
handling for non-square matrices.
- Fixed
seq()
length computation for edge cases.
- Resolved issues in
transformencode
for federated backends.
- Addressed edge cases in matrix operations:
- Sparse-dense matrix multiplication.
- Incremental slice-line pruning.
- Fixed federated operation bugs:
- Improved robustness for federated weighted covariance.
- Resolved flaky tests in federated primitives.
Code Quality and Testing
- Improved test coverage:
- Simplification rewrites, federated operations, and Python APIs.
- Added new tests for matrix operations and compression.
- Refactored and cleaned up code for better maintainability and performance.
Documentation
- Updated Python API documentation with examples for new built-ins.
- Improved overall documentation for better usability.
Dependency Updates
- Updated library versions and dependencies:
- Bumped
codecov-action
to the latest version.
- Updated other dependencies for compatibility and security.
Miscellaneous
- Added new rewrite rules for simplifications:
pushdown-sum
and rm-reorg
for better performance.
- Introduced new I/O support for cloud-optimized GeoTIFF files.
- Added new determinant function and kernels for matrix operations.