Skip to content

REFNE is only ever tested against a random forest, and the test block asserts nothing #96

Description

@eduardstan

src/Refne/README.md describes what the module is:

REFNE (Rule Extraction From Neural Network Ensemble) is a Julia implementation for extracting
interpretable rules from trained neural network ensembles using decision tree approximation.

and cites Zhou et al., Extracting Symbolic Rules from Trained Neural Network Ensembles. It is wired
in properly — REFNERuleExtractor <: RuleExtractor at src/rule-extraction.jl:130, exported, with
extractrules dispatching to refne.

But the only place it is exercised is test/rule_extraction.jl:96-101:

extractor=REFNERuleExtractor()

Xmin = map(minimum, eachcol(Xc[test, :]))
Xmax = map(maximum, eachcol(Xc[test, :]))
extracted_rules = RuleExtraction.extractrules(extractor, solem_rf, Xmin, Xmax; L=2)

solem_rf is a random forest. The neural-network-ensemble rule extractor has never been given a
neural network ensemble.

Two separate problems, and they are worth separating:

1. The block asserts nothing

There is no @test in it. It calls extractrules, binds the result, and moves on — so it is a smoke
call that proves the function does not throw, not a test that proves it does anything. extracted_rules
is then immediately rebound by the TREPAN block below. The cheapest useful fix is an assertion about
the returned decision set: non-empty, well-formed, and — the one that actually matters — a measured
fidelity
against the model it approximated. A rule extraction whose agreement with its source is
unmeasured is not yet an explanation of anything.

2. It has never met its actual subject

This is not a criticism of the implementation; it is a gap in the ecosystem. There is no neural model
type in Sole, so there was nothing to hand it. Running REFNE on a forest exercises the sampling and
tree-fitting path, which is genuinely useful, but it leaves untested the case the method exists for —
and it is the harder case, because uniform sampling in a high-dimensional raw input space is where
this family of methods is weakest.

We have opened SoleModels#79 proposing a
first-class neural leaf, with a Concept Bottleneck Model as the first instance. The CBM case is the
interesting one for REFNE specifically: over a concept bottleneck the sampling domain is a few dozen
named, bounded dimensions rather than raw input space, which is the regime Zhou's method was designed
for.

Suggested order

  1. Add the assertion and a fidelity measurement to the existing block — small, independent, valuable
    on its own, and it does not wait on anything.
  2. Once a neural leaf exists upstream, add a genuine neural-ensemble case.

Happy to do either or both. Step 1 is also a reasonable first contribution for someone new to the
package, if you would rather keep it available for that.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions