Software & Corpora

MaxEntLab

MaxEntLab is a Maximum Entropy constraint-based modeling tool. You give it a tableau (inputs, candidate outputs, observed frequencies, and constraint violations) and it fits the constraint weights that best reproduce the observed distribution.

It handles categorical and variable data, and it supports hidden structure: when several candidates share the same overt output, their probabilities are summed automatically, so a model can be fitted to data where the structure is not directly observable.

The app runs entirely in the browser.

Launch the app ↗ Source code on GitHub ↗

What it does

• Fits constraint weights by L-BFGS-B, with L1, L2 or no regularization.
• Models hidden structure.
• Normalizes frequencies within each tableau, so probabilities, percentages and raw counts can be used.
• Shows the fitted tableau with harmony, predicted and observed probabilities, and error.
• Lets you edit any weight manually and observe the predicted distribution change.
• Sorts, searches and filters the tableau, and downloads the result as a tab-separated file.

Input file format

A tab-separated TXT file (or a comma-separated CSV) with these columns:

• input — the underlying form
• output — the surface candidates
• hidden — optional; include it to model hidden structure
• probability — observed frequency, as a probability, a percentage or a raw count
• one further column per constraint, holding violation counts

Sample input files are available from the link inside the app, and in the repository.

Documentation

Full documentation (the model and its objective, every option in the interface, the output columns, and how to run MaxEntLab from the command line or from your own Python code) is in the repository: github.com/alingwist/maxentlab

Citation

• Nirheche, Ali. 2026. MaxEntLab: A Maximum Entropy constraint-based modeling tool. Amherst, MA: University of Massachusetts Amherst. https://doi.org/10.5281/zenodo.22752925

MaxEnt with Hidden Structure in R

An interactive Shiny application designed to help linguists generate phonological grammars (weights) using a Maximum Entropy model. The app is built on top of the HGR model developed by Staubs (2011). HGR finds solutions for learning problems (with or without hidden structure), generates distributions over forms, and performs online learning simulations.

Advisor: Joe Pater.

Launch the app ↗

Input file format

The input file should be a CSV or TXT file with the following columns:

• input — input forms
• output — output forms
• probability — observed probabilities or raw frequencies
• Additional columns, one per constraint.

Using the Shiny app

1. Upload your data file. Use the "Choose Input File" button to upload a CSV or TXT file.
2. Select input type. Choose between "Probabilities" and "Raw Frequencies".
3. Select prior type. Choose between L2 and L1.
4. Generate grammar. Click "Generate Grammar" to produce the grammar.
5. Edit constraint weights. If needed, adjust weights in the dynamic UI and click "Update Grammar" to recalculate probabilities and errors.
6. Download outputs. Use the "Download Output" button to save the generated tableau.

Sample input files

The Shiny app includes three sample input files, available here. They illustrate different scenarios — with and without hidden structure, and candidates supplied as raw frequencies rather than normalized probabilities. These files can be uploaded directly into the app.

Citations

• Staubs, Robert. 2011. Harmonic Grammar in R (hgR). Software/Corpora package. Amherst, MA: University of Massachusetts Amherst. github.com/rstaubs/maxent-hidden-structure
• Nirheche, Ali. 2024. Shiny App for MaxEnt with Hidden Structure. Shiny application. Amherst, MA: University of Massachusetts Amherst. alingwist.shinyapps.io/HGR_app

Acknowledgments

This research was supported by the National Science Foundation grant BCS-2140826 to the University of Massachusetts Amherst.