Uses the project-discipline-guides SKILL to remove "common" knowledge from discipline guides - #808
Uses the project-discipline-guides SKILL to remove "common" knowledge from discipline guides#808micahjo7 wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
As discussed offline, I'm uncomfortable to wholesale remove content from these guides without more data first
|
I’ll need to preface this with the disclaimer that I've mainly looked at the css, html, and forms guides but from a quick look this doesn't seem to work very well, at least for those… 😕 It removed many things that agents don't actually do in practice and merged sentences in ways that completely changed the meaning (e.g. the bit about I hate to be a party pooper, especially having said for months that we need to remove guidance that agents already follow, which is exactly what this does! I love the idea, but I suspect the approach has a fundamental flaw: If you ask agents about best practices directly they will give you a lot of things they don't actually apply in practice because knowing about a feature and actually using it are different things. I don't have a concrete counter-proposal but I think to remove guidance from hand-authored guides in an automated way, we'd need to be testing that that guidance being removed is actually followed by agents, not just known. …Which is basically what the current evals system does. Perhaps what we'd need is a very lightweight version of that. 🤔 I think this can still be useful:
|
The SKILL was developed in #748.
mirror.mdfile.mirror.mdfile content is compared with the discipline guide (e.g.guides/accessibility/accessibility/guide.md)This way we ensure that the guides do not contain information that (most) coding agents already have know.
This PR shows the results of running this SKILL on all the discipline guides that we have checked in currently.
Note that the
webmcpguide did not have any changes (no common knowledge found).