Exposing transformations applied by `Cleaner` and `TableVectorizer` by emassoulie · Pull Request #2122 · skrub-data/skrub

emassoulie · 2026-06-04T16:17:39Z

Closes #1265
Adds a method to the Cleaner and TableVectorizer transformers enabling them to give a human-readable list of the transformations (DropUninformative, ToFloat, ToDatetime) they are set to apply once fitted.

emassoulie · 2026-06-05T14:29:18Z

I've got something going with the transformation lists for both transformers! I feel the TableVectorizer transformations could be equally relevant to expose by raw transformer type (ToFloat, ToDatetime etc, like the Cleaner), or by transformer category ("numeric", "low cardinality" etc, since they are customisable). I'm thinking of going towards the latter, but I wanted to ask for a second opinion before I go ahead.

…leaner-and-tablevectorizer-transformations

…r-transformations

emassoulie · 2026-07-16T13:19:07Z

It's starting to come together! The direction chosen is to output a string viewable in any terminal, and to list columns grouped by transformation. On this second point, I've been having thoughts on doing it the other way, adding a way of describing the pipeline each individual column goes through: does that seem relevant or does the current way work fine?

I've attached the output to TableVectorizer.list_transformations() after running fit on two different examples, one on the employee dataset and one on the custom-made dataset created for the test. Does it look like the kind of output we want?

rcap107

Thank you very much for the PR @emassoulie ! I did a review of the current material.

In general, I think we're on the right path. I think the main thing to decide is the actual format of the returned string. I think there is too much whitespace, so we may want to investigate how other symbols may make it easier to parse the output.

Other than that, good job!

rcap107 · 2026-07-17T12:18:05Z

        check_is_fitted(self, "all_outputs_")
        return np.asarray(self.all_outputs_)

+    def list_transformations(self):


This is a public method so it should have a docstring

…leaner-and-tablevectorizer-transformations

rcap107

Thanks a lot for improving the PR!

I think we're getting close to the end, though I think the code can still be improved and simplified in places.

I opened a PR targeting this branch in emassoulie#2 to propose some changes in that direction

Could you also add a mention of list_transformations in the docstring of TableVectorizer and Cleaner? Without printing the entire thing, only the first few lines followed by ... to elide the rest

rcap107 · 2026-08-20T07:48:43Z

+        """Returns a string reporting the transformations applied by the table
+        vectorizer, and the columns they are each applied to. This covers every
+        preprocessing step, each of the `numeric`, `datetime`, `low cardinality`
+        and `high cardinality` transformations and any specific transformer.


numpy formatting requires a single line

Suggested change

"""Returns a string reporting the transformations applied by the table

vectorizer, and the columns they are each applied to. This covers every

preprocessing step, each of the `numeric`, `datetime`, `low cardinality`

and `high cardinality` transformations and any specific transformer.

"""Returns a string reporting the transformations applied by the TableVectorizer \

and the columns they are each applied to.

This covers every preprocessing step, each of the `numeric`, `datetime`, `low cardinality`

and `high cardinality` transformations and any specific transformer.

rcap107 · 2026-08-20T07:56:21Z

+
+
+def _limit_cols(col_names, max_cols=10):
+    list_cols = col_names[:max_cols]


I realize I suggested this version, but thinking about it again it could be simplified further to

list_cols = col_names[:max_cols] + ["..."] if len(col_names) > max_cols else col_names

though it's not a big problem since this is wrapped into a function anyway

rcap107 · 2026-08-20T07:58:30Z

+                """
+                For each column type (numeric, datetime etc.), there is a
+                dedicated transformer in the TableVectorizer that must be
+                displayed (for instance, self.numeric = Passthrough()).
+                The corresponding attribute is therefore fetched
+                and its class name printed.
+                """


I'd rather have the regular comments with # even though it's a multi-line comment. It's how we do it in the rest of the codebase, and this is being rendered as a string so at a glance I was wondering if it would be printed

rcap107 · 2026-08-20T08:21:07Z

        check_is_fitted(self, "all_outputs_")
        return np.asarray(self.all_outputs_)

+    def list_transformations(self, max_cols=10):


this needs to have a docstring

rcap107 · 2026-08-20T09:04:59Z

+                )
+
+        t_post = self._postprocessors[0]
+        postprocessing_transformations += (


maybe we can remove this section on postprocessing, since it's done to all columns anyway

it should be mentioned clearly in the docstring (I realize now it's not)

rcap107 · 2026-08-20T12:58:06Z

+
+
+def test_list_transformations(df_module):
+    def list_category(line_name, key, column_type="", with_specific=True, max_cols=3):


please add a comment here to explain what the function is doing

rcap107 · 2026-08-20T12:59:07Z

+
+        return header + full_list
+
+    passthrough_line = [


please add a small comment explaining what the test dataset looks like

rcap107 · 2026-08-20T13:12:03Z

+            All float columns"""
+    assert vectorizer_output == expected_vectorizer_output
+
+    vectorizer = Cleaner(drop_if_constant=True)


the cleaner should be moved to a separate test, or the test should be parametrized to have both the cleaner and the tablevectorizer

in this case it may be simpler to have two separate tests, though that means repeating a lot of the code

either way, list_category should be moved outside of the test

the reason I prefer having two separate tests (or a parametrized test) is that I was debugging this, and the test was failing on the Cleaner part

I did not notice that was the case because the diff was very long, so I was looking for the failure in the part about the TableVectorizer when it was in the Cleaner

rcap107 · 2026-08-20T13:12:20Z

+        + list_category("DropUninformative", "uninformative", with_specific=False)
+        + list_category("Datetime", "datetime", with_specific=False)
+    )
+    """


same comment about using # rather than triple quotes

rcap107 · 2026-08-20T13:12:33Z

+    Datetime (1 columns):
+            - datetime
+    """
+    assert cleaner_output == expected_cleaner_output


same comment about testing line by line

Created basic _list_transformation function

0a22b31

Eloi Massoulié and others added 16 commits June 10, 2026 14:25

Rough version of list_transformations

e93c7ec

Merge remote-tracking branch 'upstream/main' into issue-1265-expose-c…

3e1832c

…leaner-and-tablevectorizer-transformations

Removed pattern matching for custom transformers

36a6e71

Introduced basic test and simplified transformer list

1270c71

formatting and repr fix

054ddbf

Merge remote-tracking branch 'upstream/main' into issue-1265-expose-c…

dffb2d2

…leaner-and-tablevectorizer-transformations

Added exception for postprocessors

4b59efd

Proper empty set checks

11b6318

Merge remote-tracking branch 'upstream/main' into issue-1265-expose-c…

d2c1ef1

…leaner-and-tablevectorizer-transformations

Better indentation on column lists, and rough test

26abe8b

More specific test

35195e2

Improved test coverage

0bb9b57

Improved formatting for string output

7d891c1

Merge remote-tracking branch 'upstream/main' into issue-1265-expose-c…

bcf72cc

…leaner-and-tablevectorizer-transformations

Changelog

2c698bf

Merge branch 'main' into issue-1265-expose-cleaner-and-tablevectorize…

4b34cd5

…r-transformations

emassoulie marked this pull request as ready for review July 16, 2026 12:53

Eloi Massoulié added 2 commits July 16, 2026 15:27

Updated tests

6d3f6c5

Fixed typo

4bbcb87

rcap107 reviewed Jul 17, 2026

View reviewed changes

Eloi Massoulié added 2 commits July 28, 2026 12:33

Cleanup and test update

b3580c3

Merge remote-tracking branch 'upstream/main' into issue-1265-expose-c…

2b411e0

…leaner-and-tablevectorizer-transformations

rcap107 requested changes Aug 20, 2026

View reviewed changes



		def _limit_cols(col_names, max_cols=10):
		list_cols = col_names[:max_cols]



		def test_list_transformations(df_module):
		def list_category(line_name, key, column_type="", with_specific=True, max_cols=3):

Uh oh!

Conversation

emassoulie commented Jun 4, 2026

Uh oh!

emassoulie commented Jun 5, 2026

Uh oh!

emassoulie commented Jul 16, 2026

Uh oh!

rcap107 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rcap107 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants