Skip to content

[SYSTEMDS-3986] Create class for Sparsity-based Optimization - #2630

Closed
ywcb00 wants to merge 4 commits into
apache:mainfrom
ywcb00:feat/recompile/sparsity/rewrite/mmchain
Closed

ywcb00 wants to merge 4 commits into
apache:mainfrom
ywcb00:feat/recompile/sparsity/rewrite/mmchain

Conversation

@ywcb00

@ywcb00 ywcb00 commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Hello there,
this PR adds a separate class for sparsity-based optimizations and for related pre-computations to estimate sparsity. It is appended to the dynamic rewrites.
For now, the sparsity optimizers replicates the sparsity-based matrix multiplication rewrites. In the future, this can be extended to complete re-creation of the computation graph.
All the best,
David

@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.82353% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.03%. Comparing base (6eb4e16) to head (c500b5d).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...g/apache/sysds/hops/rewrite/SparsityOptimizer.java 82.81% 3 Missing and 8 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2630      +/-   ##
============================================
- Coverage     72.04%   72.03%   -0.01%     
- Complexity    51167    51173       +6     
============================================
  Files          2100     2101       +1     
  Lines        228681   228744      +63     
  Branches      38640    38657      +17     
============================================
+ Hits         164746   164787      +41     
- Misses        52771    52777       +6     
- Partials      11164    11180      +16     
Flag Coverage Δ
java 71.50% <83.82%> (+<0.01%) ⬆️
python 75.59% <ø> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

…te class sparsity optimizer to perform complete rewrites of isolated mm chain statement blocks
…zer to being a hop rewrite rule instead of statement block rewrite

	NOTE: hop rewrite rules are also executed after computing the previous statementblock

feat(main/hops/rewrite/SparsityOptimizer.java): replicate the optimization from normal sparsity-based mm chain rewrite
	create the method to "sculpt" a mm chain, which is allowed to completely rewrite the mm chain and ingest a new hop structure

feat(test/functions/rewrite/SparsityOptimizerTest.java): create preliminary junit test for the sparsity optimizer
…ilar to the mm chain rewrite class

chore(test/functions/rewrite/SparsityOptimizerTest.java): set the estimator type explicitly
@ywcb00
ywcb00 force-pushed the feat/recompile/sparsity/rewrite/mmchain branch from 1d437c3 to c500b5d Compare November 4, 2026 08:44
@ywcb00 ywcb00 closed this in 2d0ad72 Nov 4, 2026
@github-project-automation github-project-automation Bot moved this from In Progress to Done in SystemDS PR Queue Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant