ec1f89217a
Full Mattermost server source with integrated Community Enterprise features. Includes vendor directory for offline/air-gapped builds. Structure: - enterprise-impl/: Enterprise feature implementations - enterprise-community/: Init files that register implementations - enterprise/: Bridge imports (community_imports.go) - vendor/: All dependencies for offline builds Build (online): go build ./cmd/mattermost Build (offline/air-gapped): go build -mod=vendor ./cmd/mattermost 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| .gitignore | ||
| classifyParagraphs.go | ||
| defaultTemplate.go | ||
| detailedTemplate.go | ||
| htmlRenderer.go | ||
| LICENSE | ||
| paragraphObjectModel.go | ||
| preprocess.go | ||
| reader.go | ||
| README.md | ||
| reviseParagraphClassification.go | ||
| stoplists.go | ||
| TODO.txt | ||
| writer.go | ||
justext
A Go package that implements the JusText boilerplate removal algorithm (http://code.google.com/p/justext/)
Install
go get github.com/JalfResi/justext
And import:
import "github.com/JalfResi/justext"
Usage
Supports all stoplist files available at http://code.google.com/p/justext/source/browse/#svn%2Ftrunk%2Fjustext%2Fstoplists
Justext expects valid HTML; it is your responsability to ensure that valid HTML is passed to Justext. To make things easier I have written a CGO wrapper around libtidy which you can find here: github.com/JalfResi/GoTidy In the future, once exp/html is part of the standard packages I will refactor JusText to accept only valid HTML documents/strings.
Justext use the reader-writer idiom, alowing you to setup the reader with a common configuration and just pump out articles to the writer.
Example usage:
// Create a justext reader from another reader
reader := justext.NewReader(os.Stdin)
// Configure the reader
reader.LengthLow = 70
reader.LengthHigh = 200
reader.Stoplist = stoplist // The stoplist map[string]bool
reader.StopwordsLow = 0.3
reader.StopwordsHigh = 0.32
reader.MaxLinkDensity = 0.2
reader.MaxHeadingDistance = 200
reader.NoHeadings = false
// Read from the reader to generate a paragraph set
paragraphSet, _ := reader.ReadAll()
// Create a writer from another writer
writer := justext.NewWriter(os.Stdout)
// Write the paragraph set to the writer
writer.WriteAll(paragraphSet)