Repository navigation

swan

Public

forked from thatguystone/swan

About

An implementation of the Goose HTML Content / Article Extractor algorithm in golang

Custom properties

0 stars

0 watching

Report repository

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
misc/git		misc/git
test_data		test_data
update		update
LICENSE		LICENSE
README.md		README.md
article.go		article.go
article_test.go		article_test.go
cleanup.go		cleanup.go
extract_authors.go		extract_authors.go
extract_authors_test.go		extract_authors_test.go
extract_comic.go		extract_comic.go
extract_content.go		extract_content.go
extract_content_test.go		extract_content_test.go
extract_images.go		extract_images.go
extract_images_test.go		extract_images_test.go
extract_links.go		extract_links.go
extract_links_test.go		extract_links_test.go
extract_metas.go		extract_metas.go
extract_metas_test.go		extract_metas_test.go
extract_publishdate.go		extract_publishdate.go
extract_publishdate_test.go		extract_publishdate_test.go
extract_tags.go		extract_tags.go
extract_tags_test.go		extract_tags_test.go
extract_test.go		extract_test.go
extract_title.go		extract_title.go
extract_title_test.go		extract_title_test.go
extract_top_node.go		extract_top_node.go
extract_videos.go		extract_videos.go
extract_videos_test.go		extract_videos_test.go
go.mod		go.mod
go.sum		go.sum
http.go		http.go
logo.png		logo.png
logo.xcf		logo.xcf
stopwords.go		stopwords.go
stopwords_list.go		stopwords_list.go
stopwords_test.go		stopwords_test.go
swan.go		swan.go
swan_test.go		swan_test.go
utf8.go		utf8.go
utf8_test.go		utf8_test.go

Repository files navigation

Swan

An implementation of the Goose HTML Content / Article Extractor algorithm in golang.

Swan allows you to extract cleaned up text and HTML content from any webpage by removing all the extra junk that so many pages have these days.

Check out the go documentation page for full usage and examples.

Features

Main content extraction from almost any source
Extract HTML content with images
Get article metadata, publish dates, and a lot more
Recognize different content types and apply special extractions (currently only recognizes comic sites and normal sites)

Planned

Inline videos into HTML content when found in an article
Recognize news sources and extract corresponding video / audio content
Recognize and extract more types of content
An interesting idea: buriy/python-readability#57 (comment)

About

An implementation of the Goose HTML Content / Article Extractor algorithm in golang

Custom properties

0 stars

0 watching

Report repository

Releases

Packages

Contributors

Languages