[REVIEW]: BayesCog: A freely available course in Bayesian statistics and hierarchical Bayesian modeling for psychological science

[editorialbot]

Submitting author: @sohaamir (Aamir Sohail)
Repository: https://github.com/Alpn-Lab/BayesCog
Branch with paper.md (empty if default branch):
Version: v1.1.0
Editor: @magsol
Reviewers: @aloctavodia, @DrCatHicks
Archive: Pending
Paper kind: learning module

Status

Status badge code:

HTML: <a href="https://jose.theoj.org/papers/647e34f4a502ce670825a7621f67441a"><img src="https://jose.theoj.org/papers/647e34f4a502ce670825a7621f67441a/status.svg"></a>
Markdown: [![status](https://jose.theoj.org/papers/647e34f4a502ce670825a7621f67441a/status.svg)](https://jose.theoj.org/papers/647e34f4a502ce670825a7621f67441a)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@aloctavodia & @DrCatHicks, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://openjournals.readthedocs.io/en/jose/reviewer_guidelines.html. Any questions/concerns please let @magsol know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @DrCatHicks

📝 Checklist for @aloctavodia

[editorialbot]

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

[editorialbot]

Repository Analysis Report

Code breakdown

github.com/AlDanial/cloc v 1.98  T=0.05 s (1265.9 files/s, 181718.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
R                               40           1081           1065           3142
Text                            13              0              0           2418
TeX                              2             96              0            805
Markdown                         3            128              6            313
YAML                             4             15              7            217
Lua                              2             29             41            201
CSS                              1             17              9             59
Dockerfile                       1              9             34             37
JSON                             1              0              0             19
JavaScript                       1              0             12              1
-------------------------------------------------------------------------------
SUM:                            68           1375           1174           7212
-------------------------------------------------------------------------------

Commit Count by Author

68	Aamir Sohail
     7	sohaamir
     6	allcontributors[bot]
     1	Zhi-Lin Su

Repository History

Repository age: First commit on January 07, 2025

Code distribution (top 3 48-hour windows):

Period	Insertions	% of Total	Signal	View Changes
Jan 06, 2025 - Jan 08, 2025	3054	41.5%	🟡	View diff
Jan 23, 2025 - Jan 25, 2025	2110	28.7%	🟡	View commit
Jul 10, 2025 - Jul 12, 2025	1133	15.4%	🟢	View diff

GitHub Activity Metrics

Metric	Count
GitHub stars	4
Forks	2
GitHub contributors	3
Releases	2
Total issues	1
Total pull requests	4
Unique commenters/reviewers (excluding authors)	1

[editorialbot]

Paper file info:

🚨 Wordcount for paper.md is 2075

✅ The paper includes a Statement of need section

[editorialbot]

License info:

🔴 License found: Creative Commons Attribution Share Alike 4.0 International (Not OSI approved)

[editorialbot]

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

[editorialbot]

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

✅ OK DOIs

- 10.5281/zenodo.15804715 is OK

🟡 SKIP DOIs

- No DOI given, and none found for title: Stan: A Probabilistic Programming Language
- No DOI given, and none found for title: DartBrains: An online open access resource for lea...
- No DOI given, and none found for title: Reinforcement learning: the good, the bad and the ...
- No DOI given, and none found for title: Computational Modeling of Cognition and Behavior
- No DOI given, and none found for title: The computational challenge of social learning
- No DOI given, and none found for title: The promise of a model-based psychiatry: building ...
- No DOI given, and none found for title: Doing Bayesian Data Analysis: A Tutorial with R, J...
- No DOI given, and none found for title: A Student’s Guide to Bayesian Statistics
- No DOI given, and none found for title: Neural basis of reinforcement learning and decisio...
- No DOI given, and none found for title: Bayesian Cognitive Modeling: A Practical Course
- No DOI given, and none found for title: Computational modelling of social cognition and be...
- No DOI given, and none found for title: Vision: A computational investigation into the hum...
- No DOI given, and none found for title: Statistical Rethinking: A Bayesian Course with Exa...
- No DOI given, and none found for title: Reinforcement learning in the brain
- No DOI given, and none found for title: NivStan
- No DOI given, and none found for title: Models of Learning
- No DOI given, and none found for title: A Hitchhiker’s Guide to Bayesian Hierarchical Drif...
- No DOI given, and none found for title: A Brain-To-Brain Mechanism for Social Transmission...
- No DOI given, and none found for title: JAGS: A program for analysis of Bayesian graphical...
- No DOI given, and none found for title: A Theory of Pavlovian Conditioning: Variations in ...
- No DOI given, and none found for title: RStudio: Integrated Development Environment for R
- No DOI given, and none found for title: RStan: the R interface to Stan
- No DOI given, and none found for title: ggplot2: Elegant Graphics for Data Analysis

❌ MISSING DOIs

- 10.1101/064287 may be a valid DOI for title: Revealing Neurocomputational Mechanisms of Reinfor...
- 10.1002/wcs.1458 may be a valid DOI for title: Bayesian statistical approaches to evaluating cogn...
- 10.31234/osf.io/rtgew may be a valid DOI for title: Troubleshooting Bayesian cognitive models.
- 10.1371/journal.pbio.3000908 may be a valid DOI for title: Modeling flexible behavior in childhood to adultho...
- 10.1016/j.neuron.2011.02.027 may be a valid DOI for title: Model-based influences on humans’ choices and stri...
- 10.1016/j.bpsc.2015.11.004 may be a valid DOI for title: Different Ways of Linking Behavioral and Neural Da...
- 10.31234/osf.io/e7kwx may be a valid DOI for title: What do reinforcement learning models measure? Int...
- 10.1002/wcs.57 may be a valid DOI for title: Model-based approaches to neuroimaging: combining ...
- 10.1177/1745691620970585 may be a valid DOI for title: How Computational Modeling Can Force Theory Buildi...
- 10.1038/nn.4238 may be a valid DOI for title: Computational psychiatry as a bridge from neurosci...
- 10.1038/s41386-023-01570-y may be a valid DOI for title: Testosterone eliminates strategic prosocial behavi...
- 10.1016/j.jmp.2010.08.013 may be a valid DOI for title: How cognitive modeling can benefit from hierarchic...
- 10.1016/j.biopsych.2017.07.016 may be a valid DOI for title: Theory-Based Computational Psychiatry
- 10.1016/j.tics.2017.03.011 may be a valid DOI for title: The Importance of Falsification in Computational C...
- 10.1016/j.cub.2021.11.027 may be a valid DOI for title: Building better theories
- 10.21105/jose.00146 may be a valid DOI for title: Computational models of human social behavior and ...
- 10.31234/osf.io/e437b may be a valid DOI for title: Putting Psychology to the Test: Rethinking Model E...
- 10.1093/psyrad/kkae010 may be a valid DOI for title: Informing the treatment of social anxiety disorder...
- 10.1007/s12144-025-07438-2 may be a valid DOI for title: Using large language models to facilitate academic...
- 10.31234/osf.io/gj482_v1 may be a valid DOI for title: Recalibrating Academic Expertise in the Age of Gen...
- 10.1093/psyrad/kkaf011 may be a valid DOI for title: Interpersonal computational modelling of social sy...
- 10.32614/cran.package.renv may be a valid DOI for title: renv: Project environments
- 10.32614/cran.package.loo may be a valid DOI for title: loo: Efficient leave-one-out cross-validation and ...
- 10.31234/osf.io/46mbn may be a valid DOI for title: Ten simple rules for the computational modeling of...
- 10.1093/scan/nsaa089 may be a valid DOI for title: Using reinforcement learning models in social neur...

❌ INVALID DOIs

- None

[DrCatHicks]

Review checklist for @DrCatHicks

Conflict of interest

• As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSE code of conduct.

General checks

• Repository: Is the source for this learning module available at the https://github.com/Alpn-Lab/BayesCog?
• License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
• Version: Does the release version given match the repository release?
• Authorship: Has the submitting author (@sohaamir) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

• A statement of need: Do the authors clearly state the need for this module and who the target audience is?
• Installation instructions: Is there a clearly stated list of dependencies?
• Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

• Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
• Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
• Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
• Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
• Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Does the paper clearly state the need for this module and who the target audience is?
• Description: Does the paper describe the learning materials and sequence?
• Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
• Could someone else teach with this module, given the right expertise?
• Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[DrCatHicks]

Here are my comments, which to me are minor revisions.

Community guidelines: while there are clear notes for contributing, I noticed the support guidelines are a bit thin if someone runs into trouble installing. This is a minor note as dependencies and installation are well-described in the paper, but quickly adding a Getting Help or further troubleshooting section to README could help a reader onboard. It may be nice to consider a separate contributors.md file so that contribution guidelines do not only live in the README (this is also slightly confusing when paper states to look for contributors page”.

License should be updated to OSI approved. Creative Commons License only, but OSI license is required for code components. Suggested minor revision to add OSI approved license. License is otherwise clear and correctly in a LICENSE file.

workshops 10 and 11 are marked "WIP" and noted as “yet to be added.” I am curious to know JOSE stance on this @magsol as I’m not sure how to treat this e.g., if these are living breathing course materials, does JOSE want the paper to remain consistent with the listed workshops?

Usage: the paper has a Future Directions section but no statement about how long the course takes, what size of class it works for, or what level of instructor expertise is needed. This could be expanded in a minor revision. The paper also provides no particular guidance for instructors on how to adopt the materials (e.g., what a semester looks like, what prerequisites to set), and no discussion of student numbers or outcomes, although the README hints at impact. Some details here might strengthen a user's ability to adapt this content to the level of programming knowledge, as in below comment.

Minor note on content scope and length: R Basics workshop goes through quite a few core programming concepts quickly, and I can imagine many learners struggling here. While in the hands of an experienced teacher, one can imagine these materials being taught well, this is another note about the lack of adaptation notes, as this is a possible place to expand to the adaptation of this workshop to a programming-naive audience or signal to teachers that they may need to adapt this portion of the workshop to allow more time. Refinement to the learning objectives might also help (see below comment on making them more functionally specific), e.g., "this assumes prior x knowledge" about statistics or programming.

By my read the paper does not really tell the “story” of workshop development or author expertise, a short paragraph on this could be added to the paper. Author expertise is briefly implied by associations but functional/methodological expertise might be useful in keeping with JOSE checklist.

DOIs appear missing in references

Learning objectives are present in the paper. As another minor improvement suggestion, I believe they could be elaborated a little bit on each workshop page (e.g., workshop 4 says “By the end of this tutorial, you’ll have the foundational skills needed to implement more complex Bayesian models using Stan’s probabilistic programming framework in future workshops.” Which is not super specific in terms of concrete learning objectives). As a minor revision, I would encourage less high level descriptive and more functional learning objectives, e.g., “by the end of this workshop you will be able to perform x task in y context” to make objectives more assessable and grounded for learners.

Paper has a minor typo in paragraph directly before acknowledgments: “following the community guidelines outlined on the CONTIBUTORS page”

Positive notes: This is overall an informative contribution with significant material for learners to explore. There is a lot of thoughtful guidance to the user for use and reuse. I think these workshops are very pleasantly designed overall, and particularly like the worked examples, pedagogical exercises that allow solutions to be revealed as well as the illustrations.

[sohaamir]

Hi @DrCatHicks, thanks for the feedback! I will wait until the second reviewer has also presented their comments before submitting a revised version.

[magsol]

@DrCatHicks As a general rule, we review the current state of the submission, not its ideal state. If something is labeled as incomplete or missing, it either needs to be completed so we can review it in full, or it needs to be removed entirely from consideration.

@sohaamir This is relevant to your submission: you'll need to either fill in the "WIP" workshops to completeness, or remove them from consideration as part of this submission.

[aloctavodia]

Review checklist for @aloctavodia

Conflict of interest

• As the reviewer I confirm that I have read the JOSE conflict of interest policy and that there are no conflicts of interest for me to review this work.

Code of Conduct

• I confirm that I read and will adhere to the JOSE code of conduct.

General checks

• Repository: Is the source for this learning module available at the https://github.com/Alpn-Lab/BayesCog?
• License: Does the repository contain a plain-text LICENSE file with the contents of a standard license? (OSI-approved for code, Creative Commons for content)
• Version: Does the release version given match the repository release?
• Authorship: Has the submitting author (@sohaamir) made visible contributions to the module? Does the full list of authors seem appropriate and complete?

Documentation

• A statement of need: Do the authors clearly state the need for this module and who the target audience is?
• Installation instructions: Is there a clearly stated list of dependencies?
• Usage: Does the documentation explain how someone would adopt the module, and include examples of how to use it?
• Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the module 2) Report issues or problems with the module 3) Seek support

Pedagogy / Instructional design (Work-in-progress: reviewers, please comment!)

• Learning objectives: Does the module make the learning objectives plainly clear? (We don't require explicitly written learning objectives; only that they be evident from content and design.)
• Content scope and length: Is the content substantial for learning a given topic? Is the length of the module appropriate?
• Pedagogy: Does the module seem easy to follow? Does it observe guidance on cognitive load? (working memory limits of 7 +/- 2 chunks of information)
• Content quality: Is the writing of good quality, concise, engaging? Are the code components well crafted? Does the module seem complete?
• Instructional design: Is the instructional design deliberate and apparent? For example, exploit worked-example effects; effective multi-media use; low extraneous cognitive load.

JOSE paper

• Authors: Does the paper.md file include a list of authors with their affiliations?
• A statement of need: Does the paper clearly state the need for this module and who the target audience is?
• Description: Does the paper describe the learning materials and sequence?
• Does it describe how it has been used in the classroom or other settings, and how someone might adopt it?
• Could someone else teach with this module, given the right expertise?
• Does the paper tell the "story" of how the authors came to develop it, or what their expertise is?
• References: Do all archival references that should have a DOI list one (e.g., papers, datasets, software)?

[aloctavodia]

This is a good resource. I found the material to be engaging, easy to follow, and with concepts clearly explained, often in a few lines, which I think is good for newcomers to the field. I have opened a few issues (all minor comments) directly on the repository.