Skip to content

Tags: shurco/goClone

Tags

v1.0.5

Toggle v1.0.5's commit message

Verified

This commit was signed with the committer’s verified signature.
shurco Shurco
♻️ refactor(crawler): harden mirror pipeline and modernize backend

Changes made in this commit:

- Modified: README.md, cmd/main.go, go.mod, go.sum
- Modified: pkg/crawler/base.go, concurrency_test.go, crawler.go, crawler_test.go, css.go, init_download.go, integration_test.go
- Modified: pkg/fsutil/file.go, folder.go, fsutil_test.go
- Modified: pkg/netutil/netutil.go, netutil_test.go
- Added: pkg/crawler/asset.go
- Deleted: pkg/crawler/img.go, pkg/crawler/js.go

Key changes:

- Serialize access to discovered assets; fix O(n^2) page scheduling
- Replace log.Fatal in parsers with logging; wait for download goroutines
- Merge saveJS/saveIMG into saveAsset; package-level CSS url regexp
- netutil: context-aware Extractor, strip query in urlExtension, check size before open
- fsutil: filepath.Dir/Join, WriteOSFile returns error, truncate on mirror writes
- Serve via stdlib http.Server with graceful shutdown; update README and --serve help

v1.0.4

Toggle v1.0.4's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Merge pull request #33 from shurco/22-error

πŸ”§ fix: Update file path handling in crawler and asset loading

v1.0.3

Toggle v1.0.3's commit message

    

v1.0.2

Toggle v1.0.2's commit message

    

v1.0.1

Toggle v1.0.1's commit message

    

v1.0.0

Toggle v1.0.0's commit message