src/Refne/README.md describes what the module is:
REFNE (Rule Extraction From Neural Network Ensemble) is a Julia implementation for extracting
interpretable rules from trained neural network ensembles using decision tree approximation.
and cites Zhou et al., Extracting Symbolic Rules from Trained Neural Network Ensembles. It is wired
in properly — REFNERuleExtractor <: RuleExtractor at src/rule-extraction.jl:130, exported, with
extractrules dispatching to refne.
But the only place it is exercised is test/rule_extraction.jl:96-101:
extractor=REFNERuleExtractor()
Xmin = map(minimum, eachcol(Xc[test, :]))
Xmax = map(maximum, eachcol(Xc[test, :]))
extracted_rules = RuleExtraction.extractrules(extractor, solem_rf, Xmin, Xmax; L=2)
solem_rf is a random forest. The neural-network-ensemble rule extractor has never been given a
neural network ensemble.
Two separate problems, and they are worth separating:
1. The block asserts nothing
There is no @test in it. It calls extractrules, binds the result, and moves on — so it is a smoke
call that proves the function does not throw, not a test that proves it does anything. extracted_rules
is then immediately rebound by the TREPAN block below. The cheapest useful fix is an assertion about
the returned decision set: non-empty, well-formed, and — the one that actually matters — a measured
fidelity against the model it approximated. A rule extraction whose agreement with its source is
unmeasured is not yet an explanation of anything.
2. It has never met its actual subject
This is not a criticism of the implementation; it is a gap in the ecosystem. There is no neural model
type in Sole, so there was nothing to hand it. Running REFNE on a forest exercises the sampling and
tree-fitting path, which is genuinely useful, but it leaves untested the case the method exists for —
and it is the harder case, because uniform sampling in a high-dimensional raw input space is where
this family of methods is weakest.
We have opened SoleModels#79 proposing a
first-class neural leaf, with a Concept Bottleneck Model as the first instance. The CBM case is the
interesting one for REFNE specifically: over a concept bottleneck the sampling domain is a few dozen
named, bounded dimensions rather than raw input space, which is the regime Zhou's method was designed
for.
Suggested order
- Add the assertion and a fidelity measurement to the existing block — small, independent, valuable
on its own, and it does not wait on anything.
- Once a neural leaf exists upstream, add a genuine neural-ensemble case.
Happy to do either or both. Step 1 is also a reasonable first contribution for someone new to the
package, if you would rather keep it available for that.
src/Refne/README.mddescribes what the module is:and cites Zhou et al., Extracting Symbolic Rules from Trained Neural Network Ensembles. It is wired
in properly —
REFNERuleExtractor <: RuleExtractoratsrc/rule-extraction.jl:130, exported, withextractrulesdispatching torefne.But the only place it is exercised is
test/rule_extraction.jl:96-101:solem_rfis a random forest. The neural-network-ensemble rule extractor has never been given aneural network ensemble.
Two separate problems, and they are worth separating:
1. The block asserts nothing
There is no
@testin it. It callsextractrules, binds the result, and moves on — so it is a smokecall that proves the function does not throw, not a test that proves it does anything.
extracted_rulesis then immediately rebound by the TREPAN block below. The cheapest useful fix is an assertion about
the returned decision set: non-empty, well-formed, and — the one that actually matters — a measured
fidelity against the model it approximated. A rule extraction whose agreement with its source is
unmeasured is not yet an explanation of anything.
2. It has never met its actual subject
This is not a criticism of the implementation; it is a gap in the ecosystem. There is no neural model
type in Sole, so there was nothing to hand it. Running REFNE on a forest exercises the sampling and
tree-fitting path, which is genuinely useful, but it leaves untested the case the method exists for —
and it is the harder case, because uniform sampling in a high-dimensional raw input space is where
this family of methods is weakest.
We have opened SoleModels#79 proposing a
first-class neural leaf, with a Concept Bottleneck Model as the first instance. The CBM case is the
interesting one for REFNE specifically: over a concept bottleneck the sampling domain is a few dozen
named, bounded dimensions rather than raw input space, which is the regime Zhou's method was designed
for.
Suggested order
on its own, and it does not wait on anything.
Happy to do either or both. Step 1 is also a reasonable first contribution for someone new to the
package, if you would rather keep it available for that.