Looking for the JS/TS version? Check out LangChain.js.
pip install langchain-testsThis is a testing library for LangChain integrations. It contains the base classes for a standard set of tests.
For full documentation, see the API reference.
See our Releases and Versioning policies.
We encourage pinning your version to a specific version in order to avoid breaking your CI when we publish new tests. We recommend upgrading to the latest version periodically to make sure you have the latest tests.
Not pinning your version will ensure you always have the latest tests, but it may also break your CI if we introduce tests that your integration doesn't pass.
As an open-source project in a rapidly developing field, we are extremely open to contributions, whether it be in the form of a new feature, improved infrastructure, or better documentation.
For detailed information on how to contribute, see the Contributing Guide.
To add standard tests to an integration package (e.g., for a chat model), you need to create
ChatModelUnitTestsChatModelIntegrationTeststests/unit_tests/test_standard.py:
"""Standard LangChain interface tests"""
from typing import Type
import pytest
from langchain_core.language_models import BaseChatModel
from langchain_tests.unit_tests import ChatModelUnitTests
from langchain_parrot_chain import ChatParrotChain
class TestParrotChainStandard(ChatModelUnitTests):
@pytest.fixture
def chat_model_class(self) -> Type[BaseChatModel]:
return ChatParrotChaintests/integration_tests/test_standard.py:
"""Standard LangChain interface tests"""
from typing import Type
import pytest
from langchain_core.language_models import BaseChatModel
from langchain_tests.integration_tests import ChatModelIntegrationTests
from langchain_parrot_chain import ChatParrotChain
class TestParrotChainStandard(ChatModelIntegrationTests):
@pytest.fixture
def chat_model_class(self) -> Type[BaseChatModel]:
return ChatParrotChainThe following fixtures are configurable in the test classes. Anything not marked as required is optional.
chat_model_class (required): The class of the chat model to be testedchat_model_params: The keyword arguments to pass to the chat model constructorchat_model_has_tool_calling: Whether the chat model can call tools. By default, this is set to hasattr(chat_model_class, 'bind_tools)chat_model_has_structured_output: Whether the chat model can structured output. By default, this is set to hasattr(chat_model_class, 'with_structured_output')Custom serializer for VCR cassettes using YAML and gzip.
A custom persister for VCR that uses the CustomSerializer.
Base class for standard tests.
Embeddings tests base class.
Base class for embeddings unit tests.
Base class for chat model tests.
Base class for chat model unit tests.
Base class for testing tools.
Base class for tools unit tests.
Base class for embeddings integration tests.
Test suite for checking the read-write of a document index.
Test suite for checking the read-write of a document index.
Base class for vector store integration tests.
Test suite for checking the BaseCache API of a caching layer for LLMs.
Test suite for checking the BaseCache API of a caching layer for LLMs.
Base class for chat model integration tests.
Base class for retrievers integration tests.
Standard integration tests for a SandboxBackendProtocol implementation.
Base class for tools integration tests.
Test suite for checking the key-value API of a BaseStore.
Test suite for checking the key-value API of a BaseStore.
Return VCR configuration that every cassette will receive.
VCR config fixture.
Get the major version of Pydantic.
Works with either pydantic 1 or 2.
Apply a magic function to an input.
Calculate a magic function.
Tool for creating a customer with Unicode name.