ec1f89217a
Full Mattermost server source with integrated Community Enterprise features. Includes vendor directory for offline/air-gapped builds. Structure: - enterprise-impl/: Enterprise feature implementations - enterprise-community/: Init files that register implementations - enterprise/: Bridge imports (community_imports.go) - vendor/: All dependencies for offline builds Build (online): go build ./cmd/mattermost Build (offline/air-gapped): go build -mod=vendor ./cmd/mattermost 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
49 lines
1.6 KiB
Markdown
49 lines
1.6 KiB
Markdown
justext
|
|
=======
|
|
|
|
A Go package that implements the JusText boilerplate removal algorithm (http://code.google.com/p/justext/)
|
|
|
|
## Install
|
|
|
|
go get github.com/JalfResi/justext
|
|
|
|
And import:
|
|
|
|
import "github.com/JalfResi/justext"
|
|
|
|
## Usage
|
|
|
|
Supports all stoplist files available at http://code.google.com/p/justext/source/browse/#svn%2Ftrunk%2Fjustext%2Fstoplists
|
|
|
|
Justext expects valid HTML; it is your responsability to ensure that valid HTML is passed to Justext. To make things easier
|
|
I have written a CGO wrapper around libtidy which you can find here: [github.com/JalfResi/GoTidy](https://github.com/JalfResi/GoTidy)
|
|
In the future, once exp/html is part of the standard packages I will refactor JusText to accept only valid HTML documents/strings.
|
|
|
|
Justext use the reader-writer idiom, alowing you to setup the reader with a common configuration and just pump out
|
|
articles to the writer.
|
|
|
|
Example usage:
|
|
|
|
// Create a justext reader from another reader
|
|
reader := justext.NewReader(os.Stdin)
|
|
|
|
// Configure the reader
|
|
reader.LengthLow = 70
|
|
reader.LengthHigh = 200
|
|
reader.Stoplist = stoplist // The stoplist map[string]bool
|
|
reader.StopwordsLow = 0.3
|
|
reader.StopwordsHigh = 0.32
|
|
reader.MaxLinkDensity = 0.2
|
|
reader.MaxHeadingDistance = 200
|
|
reader.NoHeadings = false
|
|
|
|
// Read from the reader to generate a paragraph set
|
|
paragraphSet, _ := reader.ReadAll()
|
|
|
|
// Create a writer from another writer
|
|
writer := justext.NewWriter(os.Stdout)
|
|
// Write the paragraph set to the writer
|
|
writer.WriteAll(paragraphSet)
|
|
|
|
|