Merge pull request #2025 from weaviate/changelog_4_21_1

Add changelog for v4.21.1

Support `1.37` features (#1988)

* Refactor hybrid queries to use `alpha_param` and remove `0.7` default in favour of `None`

* Remove mistakenly commited local change to `regen.sh`

* Update logic to use new proto message

* Change formatting

* Tidy version check code

* Parse correct default for BC if server < 1.36

* Update CI image

* Fix wrong version comparison

* Fix typo in ci

* Update CI image

* Remove client-side default from aggregate queries

* Update ver check and CI tags

* Remove test of lazy loading shards

* Refactor client test for new server lazy shard loading

* Debug failing ci test

* Remove outdated lazy shard load test

* Update CI images

* Add per-test timeouts and stack dump on timeout

* Reduce per-test timeout to 5 mins

* Fix inc backups test

* Add server version check for incremental backups

* Remove comment

* Hard kill the process on timeout detection

* Timeout putting the sentinel to avoid deadlocking

* Handle the timeout of sentinel pushing gracefully

* feat: add TextAnalyzerConfig for ASCII folding in text properties

* refactor: ruff format

* feat: add min version check

* feat: update TextAnalyzerConfig docstring for ascii_fold attributes

* feat: add asciiFold check in _text_analyzer_from_config function

* test: fix ASCII folding tests

* feat: add support for stopword presets in inverted index configuration and text analyzer

* test: added live and config tests

* refactor: improve docstrings for stopword presets and asciiFold tests

* refactor: simplify _any_property_has_text_analyzer function using _property_has_text_analyzer

* test: remove redundant insertion ascii fold tests from test_collection_config

* test: add stopwords roundtrip test for collection configuration

* feat: add model validator to enforce asciiFoldIgnore constraints in TextAnalyzerConfig

* feat: add factory class for text analyzer configurations with ASCII folding support

* refactor: update TextAnalyzerConfig usage to new Configure class methods

* test: remove redundant line in stopword presets merge test

* refactor: use factory pattern

* Add MCP permission

* refactor: format text analyzer configuration for better readability

* refactor: remove server side behavior tests

* test: add stopword presets roundtrip tests for Weaviate collections

* Fix formatting

* refactor: remove unnecessary stopword preset coercion from _TextAnalyzerConfigCreate

* refactor: replace custom text analyzer method with a direct function call

* chore: remove unused deprecated import from config.py

* chore: update WEAVIATE_137 version to 1.37.0-rc.1-578c4eb in workflow

* refactor: update text analyzer method to use new static method in Configure

* test: add stopwords roundtrip test with ASCII folding configuration

* Add query profiling

* Reformatted

* Skip test for lower versions

* feat: add tokenizer module with sync and async support, including integration tests

* Add support for collection export endpoint

* Small cleanup after review

* Rename ENum

* adapt to latest version

* Update UX

* Remove export path parameter

* Self-review of changes

* Review fixes

* Add version guard for export integration tests

* Update to latest image

* Lowercase export ID

* Enforce kwargs for export

* Fix tests

* Fix tests

* Add negative assertions

* refactor: names don't shadow existing

* fix: add version gate

* refactor: update tokenization type to use Tokenization enum in TokenizeResult and related tests

* refactor: models

* refactor: move tokenize property to class config

* fix: remove trailing whitespace in __init__.py

* test: add version gate for Weaviate >= 1.37.0 in tokenization tests

* feat: add support for blobHash property type

* Add full_with_profile

* Refactor RBAC permissions

* Bump Weaviate version

* refactor: tokenization executor and models to support stopword configurations and improve response handling

* fix: update Weaviate 1.37.1 version to include specific build identifier

* Reduce timeouts in batch tests

* fix: update Weaviate 1.37.1 version to include architecture suffix

* fix: refactor tokenization tests to use parameterized cases for improved readability and maintainability

* fix: update Weaviate 1.37.1 version and enhance tokenization tests with new fixtures

* refactor: ruff format

* test: refactor output types and tests to config

* refactor: remove unused imports in tokenization models and format

* Use public classes for .text endpoint

* Add overloads for exclusivity of stopwrods

* Accept collection config classes as stopwords

* Improve docstring

* Hook up tokenization and clean up model

* Move property back to tokenization

* Add changelog

---------

Co-authored-by: André Mourão <andre.mourao@weaviate.io>
Co-authored-by: Dirk Kulawiak <dirk@semi.technology>
Co-authored-by: André Mourão <andre.b.mourao@gmail.com>
Co-authored-by: Ivan Despot <66276597+g-despot@users.noreply.github.com>
Co-authored-by: Dirk Kulawiak <dirk@weaviate.io>
Co-authored-by: Marcin Antas <antas.marcin@gmail.com>

Merge pull request #2010 from weaviate/mcp-rbac

Add MCP permission

Merge pull request #2007 from weaviate/changelog_4_20_5

Changelog + reformat

Merge pull request #1982 from weaviate/changelog_4_20_4

Add changelog for v4.20.4

Bump authlib to 1.6.9 to avoid high cve (#1975)

* Bump authlib to 1.6.9 to avoid high cve

* Reduce ver requirement in `setup.cfg`

* update changelog

Merge pull request #1973 from weaviate/changelog_v4-20-2

Changelog for v4.20.2

Merge pull request #1963 from weaviate/changelog_4_20_1

Add changelog for 4.20.1

Merge pull request #1960 from weaviate/changelog_v4-20-0

Add changelog for v4.20.0

Merge pull request #1848 from weaviate/rob/spfresh

Add Hfresh index type