add some features

[2077DevWave]

add reason into the results
add option to can choose number of testcase u want to generate

[gemini-code-assist]

Summary of Changes

Hello @2077DevWave, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the stress testing capabilities by introducing user-configurable test case generation and a more informative feedback mechanism for test results. It refactors the underlying command execution logic to allow checkers to provide specific reasons for wrong answers, which are then captured and presented to the user. These changes enhance the usability and diagnostic power of the testing environment.

Highlights

• Configurable Test Cases: Users can now specify the desired number of test cases to run directly from the webview interface, replacing the previously hardcoded value.
• Enhanced Checker Feedback: The system now captures and displays specific reasons for 'Wrong Answer' (WA) results, provided by the checker's stderr output. The checker_template.cpp has been updated with guidance on how to implement this.
• New Raw Command Execution Method: A new runRaw method has been introduced in the Executor class, allowing for direct execution of commands and capturing their raw exit code, stdout, and stderr, which is leveraged for improved checker integration.
• Data Structure Updates: The ITestRunResult and IJsonTestResult interfaces have been extended to include an optional reason field, enabling the storage and display of detailed feedback from the checker.
• Webview Path Correction: The MyPanelProvider has been updated to correctly reference the webview.html file from the src directory.
• Test Suite Refinements: Unit tests for Executor and TestRunner have been updated and refactored to cover the new runRaw method and the enhanced checker result handling.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[gemini-code-assist]

Code Review

This pull request introduces two main features: adding a 'reason' for failed test cases from the checker's output, and allowing the user to specify the number of test cases to run. The changes are generally well-implemented, but there are a few significant issues to address.

Most critically, a new runRaw function in Executor.ts introduces a command injection vulnerability by concatenating arguments into a command string. This must be fixed by using execFile. Additionally, the switch to runRaw for the checker process has removed resource limits (like timeouts), which is a regression that could cause the test runner to hang.

Other points include a significant change in testing behavior where tests no longer stop on the first failure, a potential issue with asset pathing for the packaged extension, reduced test coverage in Executor.test.ts, and a minor UI improvement for input validation in the webview.

Please review the detailed comments for suggestions on how to address these points.

[2077DevWave]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces two valuable features: the ability to specify the number of test cases to run and the inclusion of a reason for 'Wrong Answer' verdicts, which is captured from the checker's stderr. The refactoring in TestRunner to use execFile via a new runRaw method is a significant improvement for both security and code clarity. My review has identified a critical issue in the tests for the new Executor.runRaw method, where the wrong function is being mocked, leading to incorrect testing. I also found a medium-severity type-safety issue in the implementation of runRaw itself. Addressing these points will improve the robustness of the new features.