[FEATURE] Ragged Encoding. NOT WORKING YET. Only for Intermediate presentations

[Paulo-Mansano]

New classes: ColumnEncoderRagged, TransformDummySeparatedTest, TransformRaggedTest.
Modified: TfUtils, ColumnEncoder, EncoderFactory, TransformCustomTest

What changes were proposed?

• Added ColumnEncoderRagged in order to implement ragged array encoding.
• Added tests: TransformDummySeparatedTest, TransformRaggedTest.
• Modified TfUtils, ColumnEncoder, EncoderFactory just to add RAGGED as a encoding method.

How was this tested?

• Local tests not passed, still looking to make everything work.

Related Issues

• Still looking for the solutions and running tests to understand where are the problems.

[Baunsgaard]

Hi @Paulo-Mansano ,

Thanks for opening the PR!
It seems we slightly miscommunicated, since the intention is to 'modify' the existing transformations to construct ragged arrays as their metadata, not to create new transformations.

In specific we need to look at :

systemds/src/main/java/org/apache/sysds/runtime/transform/encode/Encoder.java

Line 70 in 692dcc3

voidinitMetaData(FrameBlockmeta);

Where the metadata is allocated, (we want to modify it to allocate the ragged array)

and

systemds/src/main/java/org/apache/sysds/runtime/transform/encode/Encoder.java

Line 63 in 692dcc3

FrameBlockgetMetaData(FrameBlockout);

Where the metadata frame is populated with information. (we want it to be in the form of a ragged array)

I hope this helps clarify things.

[codecov]

Codecov Report

❌ Patch coverage is 42.10526% with 55 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.94%. Comparing base (692dcc3) to head (137cf20).
⚠️ Report is 88 commits behind head on main.

Files with missing lines	Patch %	Lines
.../runtime/transform/encode/ColumnEncoderRagged.java	35.71%	48 Missing and 6 partials ⚠️
...sysds/runtime/transform/encode/EncoderFactory.java	87.50%	1 Missing ⚠️

Additional details and impacted files

@@ Coverage Diff @@## main #2282 +/- ## ============================================ - Coverage 72.95% 72.94% -0.01% - Complexity 46086 46103 +17  ============================================ Files 1479 1480 +1 Lines 172654 172744 +90 Branches 33796 33817 +21 ============================================ + Hits 125955 126007 +52 - Misses 37201 37233 +32 - Partials 9498 9504 +6 

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.