Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(langchain): fix flaky cassette tests, skipping logic #9768

Merged
merged 29 commits into from
Jul 30, 2024
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c0d998a
Remove flaky markers
Yun-Kim Jul 9, 2024
c139f43
tmp change to langchain integration to only trigger langchain tests
Yun-Kim Jul 9, 2024
5c4fdb6
Change pytest skip markers?
Yun-Kim Jul 9, 2024
f6d7302
small cleanup of test file
Yun-Kim Jul 10, 2024
d16172a
Add verbose option to pytest run to see which tests are being skipped
Yun-Kim Jul 11, 2024
812dc5d
Pin vcrpy, regenerate some cassettes
Yun-Kim Jul 17, 2024
785e252
Merge branch 'main' into yunkim/langchain-fix-flaky
Yun-Kim Jul 19, 2024
1b35dc4
Merge branch 'main' into yunkim/langchain-fix-flaky
Yun-Kim Jul 25, 2024
232b8f7
Remove flaky
Yun-Kim Jul 25, 2024
1bde40d
Fix langchain version check in test_langchain_llmobs.py
Yun-Kim Jul 26, 2024
bd81aff
Investigate sys.version_info issue on 3.9
Yun-Kim Jul 27, 2024
e74c9f8
Fix bedrock trace struct tests
Yun-Kim Jul 27, 2024
d35e50e
Use langchain and py39 version gating correctly
Yun-Kim Jul 27, 2024
6ea5288
Revert commenting openai test api key
Yun-Kim Jul 27, 2024
90fa6d0
Remove streaming tests, fix skip check
Yun-Kim Jul 27, 2024
13dfb24
Regenerate remaining cassettes (minus pinecone) with pinned vcrpy
Yun-Kim Jul 27, 2024
2d47347
Fix fmt, test anthropic api key
Yun-Kim Jul 27, 2024
77f6b4c
Merge branch 'main' into yunkim/langchain-fix-flaky
Yun-Kim Jul 27, 2024
da22b73
Regenerate lcel tool cassettes with pinned vcrpy
Yun-Kim Jul 27, 2024
d4a3e7b
Regenerate pinecone test cassette
Yun-Kim Jul 27, 2024
da574bc
Skip langchain 0.1.20 tests for ai21 and cohere, pin vcrpy to 6.0.1 f…
Yun-Kim Jul 27, 2024
fc9277a
Update riot lockfiles
Yun-Kim Jul 27, 2024
43af5a1
Revert pytest-asyncio dep to main branch
Yun-Kim Jul 27, 2024
c061dce
Change py39 skip marker
Yun-Kim Jul 29, 2024
2c5b38c
Remove skip marker for langchain community tmp
Yun-Kim Jul 29, 2024
6ef4e9a
Skip ai21 tests for python3.9
Yun-Kim Jul 29, 2024
69cf692
Fix snapshot test
Yun-Kim Jul 29, 2024
3b416e1
Merge branch 'main' into yunkim/langchain-fix-flaky
Yun-Kim Jul 29, 2024
a1503ab
Merge branch 'main' into yunkim/langchain-fix-flaky
Yun-Kim Jul 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/1a15059.in
# pip-compile --allow-unsafe --no-annotate .riot/requirements/11063bf.in
#
ai21==2.9.2
ai21-tokenizer==0.11.2
Expand Down Expand Up @@ -32,7 +32,7 @@ h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.24.1
huggingface-hub==0.24.2
hypothesis==6.45.0
idna==3.7
iniconfig==2.0.0
Expand Down Expand Up @@ -64,14 +64,14 @@ pluggy==1.5.0
psutil==6.0.0
pydantic==2.8.2
pydantic-core==2.20.1
pytest==8.3.1
pytest==8.3.2
pytest-asyncio==0.23.7
pytest-cov==5.0.0
pytest-mock==3.14.0
pytest-randomly==3.10.1
python-dateutil==2.9.0.post0
pyyaml==6.0.1
regex==2024.5.15
regex==2024.7.24
requests==2.32.3
s3transfer==0.10.2
sentencepiece==0.2.0
Expand All @@ -87,6 +87,6 @@ types-requests==2.32.0.20240712
typing-extensions==4.12.2
typing-inspect==0.9.0
urllib3==2.0.7
vcrpy==6.0.1
vcrpy==5.1.0
wrapt==1.16.0
yarl==1.9.4
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/1fda250.in
# pip-compile --allow-unsafe --no-annotate .riot/requirements/16c3b9f.in
#
ai21==1.3.4
aiohttp==3.9.5
Expand All @@ -23,7 +23,7 @@ filelock==3.15.4
frozenlist==1.4.1
fsspec==2024.6.1
greenlet==3.0.3
huggingface-hub==0.24.0
huggingface-hub==0.24.2
hypothesis==6.45.0
idna==3.7
importlib-metadata==6.11.0
Expand All @@ -50,14 +50,14 @@ pinecone-client==2.2.4
pluggy==1.5.0
psutil==6.0.0
pydantic==1.10.17
pytest==8.2.2
pytest==8.3.2
pytest-asyncio==0.23.7
pytest-cov==5.0.0
pytest-mock==3.14.0
pytest-randomly==3.10.1
python-dateutil==2.9.0.post0
pyyaml==6.0.1
regex==2024.5.15
regex==2024.7.24
requests==2.32.3
six==1.16.0
sniffio==1.3.1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/11f9de5.in
# pip-compile --allow-unsafe --no-annotate .riot/requirements/1761cfc.in
#
ai21==2.9.2
ai21-tokenizer==0.11.2
Expand Down Expand Up @@ -32,7 +32,7 @@ h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.24.1
huggingface-hub==0.24.2
hypothesis==6.45.0
idna==3.7
iniconfig==2.0.0
Expand All @@ -45,7 +45,7 @@ langchain-aws==0.1.6
langchain-cohere==0.1.8
langchain-core==0.2.0
langchain-openai==0.1.7
langchain-pinecone==0.1.2
langchain-pinecone==0.1.3
langchain-text-splitters==0.2.1
langsmith==0.1.93
marshmallow==3.21.3
Expand All @@ -54,25 +54,26 @@ multidict==6.0.5
mypy-extensions==1.0.0
numexpr==2.8.5
numpy==1.26.4
openai==1.37.0
openai==1.37.1
opentracing==2.4.0
orjson==3.10.6
packaging==23.2
parameterized==0.9.0
pinecone-client==4.1.2
pinecone-client==5.0.0
pinecone-plugin-inference==1.0.2
pinecone-plugin-interface==0.0.7
pluggy==1.5.0
psutil==6.0.0
pydantic==2.8.2
pydantic-core==2.20.1
pytest==8.3.1
pytest==8.3.2
pytest-asyncio==0.23.7
pytest-cov==5.0.0
pytest-mock==3.14.0
pytest-randomly==3.10.1
python-dateutil==2.9.0.post0
pyyaml==6.0.1
regex==2024.5.15
regex==2024.7.24
requests==2.32.3
s3transfer==0.10.2
sentencepiece==0.2.0
Expand All @@ -89,6 +90,6 @@ types-requests==2.32.0.20240712
typing-extensions==4.12.2
typing-inspect==0.9.0
urllib3==2.0.7
vcrpy==6.0.1
vcrpy==5.1.0
wrapt==1.16.0
yarl==1.9.4
27 changes: 14 additions & 13 deletions .riot/requirements/5425133.txt → .riot/requirements/18bc2ac.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This file is autogenerated by pip-compile with Python 3.11
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/5425133.in
# pip-compile --allow-unsafe --no-annotate .riot/requirements/18bc2ac.in
#
ai21==2.9.2
ai21-tokenizer==0.11.2
Expand All @@ -12,8 +12,8 @@ annotated-types==0.7.0
anthropic==0.31.2
anyio==4.4.0
attrs==23.2.0
boto3==1.34.146
botocore==1.34.146
boto3==1.34.149
botocore==1.34.149
certifi==2024.7.4
charset-normalizer==3.3.2
cohere==5.6.2
Expand All @@ -31,7 +31,7 @@ h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.24.1
huggingface-hub==0.24.2
hypothesis==6.45.0
idna==3.7
iniconfig==2.0.0
Expand All @@ -40,14 +40,14 @@ jmespath==1.0.1
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.11
langchain-anthropic==0.1.20
langchain-anthropic==0.1.21
langchain-aws==0.1.12
langchain-cohere==0.1.9
langchain-community==0.2.10
langchain-core==0.2.23
langchain-core==0.2.24
langchain-experimental==0.0.63
langchain-openai==0.1.17
langchain-pinecone==0.1.2
langchain-openai==0.1.19
langchain-pinecone==0.1.3
langchain-text-splitters==0.2.2
langsmith==0.1.93
marshmallow==3.21.3
Expand All @@ -56,27 +56,28 @@ multidict==6.0.5
mypy-extensions==1.0.0
numexpr==2.8.5
numpy==1.26.4
openai==1.37.0
openai==1.37.1
opentracing==2.4.0
orjson==3.10.6
packaging==24.1
pandas==2.2.2
parameterized==0.9.0
pinecone-client==4.1.2
pinecone-client==5.0.0
pinecone-plugin-inference==1.0.2
pinecone-plugin-interface==0.0.7
pluggy==1.5.0
psutil==6.0.0
pydantic==2.8.2
pydantic-core==2.20.1
pytest==8.3.1
pytest==8.3.2
pytest-asyncio==0.23.7
pytest-cov==5.0.0
pytest-mock==3.14.0
pytest-randomly==3.10.1
python-dateutil==2.9.0.post0
pytz==2024.1
pyyaml==6.0.1
regex==2024.5.15
regex==2024.7.24
requests==2.32.3
s3transfer==0.10.2
sentencepiece==0.2.0
Expand All @@ -94,6 +95,6 @@ typing-extensions==4.12.2
typing-inspect==0.9.0
tzdata==2024.1
urllib3==2.2.2
vcrpy==6.0.1
vcrpy==5.1.0
wrapt==1.16.0
yarl==1.9.4
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/1a8a6e1.in
# pip-compile --allow-unsafe --no-annotate .riot/requirements/19f2225.in
#
ai21==2.9.2
ai21-tokenizer==0.11.2
Expand Down Expand Up @@ -33,7 +33,7 @@ h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.24.1
huggingface-hub==0.24.2
hypothesis==6.45.0
idna==3.7
iniconfig==2.0.0
Expand Down Expand Up @@ -65,14 +65,14 @@ pluggy==1.5.0
psutil==6.0.0
pydantic==2.8.2
pydantic-core==2.20.1
pytest==8.3.1
pytest==8.3.2
pytest-asyncio==0.23.7
pytest-cov==5.0.0
pytest-mock==3.14.0
pytest-randomly==3.10.1
python-dateutil==2.9.0.post0
pyyaml==6.0.1
regex==2024.5.15
regex==2024.7.24
requests==2.32.3
s3transfer==0.10.2
sentencepiece==0.2.0
Expand All @@ -89,6 +89,6 @@ types-requests==2.32.0.20240712
typing-extensions==4.12.2
typing-inspect==0.9.0
urllib3==2.0.7
vcrpy==6.0.1
vcrpy==5.1.0
wrapt==1.16.0
yarl==1.9.4
27 changes: 14 additions & 13 deletions .riot/requirements/8dd750a.txt → .riot/requirements/1ec1dbf.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile --no-annotate .riot/requirements/8dd750a.in
# pip-compile --allow-unsafe --no-annotate .riot/requirements/1ec1dbf.in
#
ai21==2.9.2
ai21-tokenizer==0.11.2
Expand All @@ -13,8 +13,8 @@ anthropic==0.31.2
anyio==4.4.0
async-timeout==4.0.3
attrs==23.2.0
boto3==1.34.146
botocore==1.34.146
boto3==1.34.149
botocore==1.34.149
certifi==2024.7.4
charset-normalizer==3.3.2
cohere==5.6.2
Expand All @@ -32,7 +32,7 @@ h11==0.14.0
httpcore==1.0.5
httpx==0.27.0
httpx-sse==0.4.0
huggingface-hub==0.24.1
huggingface-hub==0.24.2
hypothesis==6.45.0
idna==3.7
iniconfig==2.0.0
Expand All @@ -41,14 +41,14 @@ jmespath==1.0.1
jsonpatch==1.33
jsonpointer==3.0.0
langchain==0.2.11
langchain-anthropic==0.1.20
langchain-anthropic==0.1.21
langchain-aws==0.1.12
langchain-cohere==0.1.9
langchain-community==0.2.10
langchain-core==0.2.23
langchain-core==0.2.24
langchain-experimental==0.0.63
langchain-openai==0.1.17
langchain-pinecone==0.1.2
langchain-openai==0.1.19
langchain-pinecone==0.1.3
langchain-text-splitters==0.2.2
langsmith==0.1.93
marshmallow==3.21.3
Expand All @@ -57,27 +57,28 @@ multidict==6.0.5
mypy-extensions==1.0.0
numexpr==2.8.5
numpy==1.26.4
openai==1.37.0
openai==1.37.1
opentracing==2.4.0
orjson==3.10.6
packaging==24.1
pandas==2.2.2
parameterized==0.9.0
pinecone-client==4.1.2
pinecone-client==5.0.0
pinecone-plugin-inference==1.0.2
pinecone-plugin-interface==0.0.7
pluggy==1.5.0
psutil==6.0.0
pydantic==2.8.2
pydantic-core==2.20.1
pytest==8.3.1
pytest==8.3.2
pytest-asyncio==0.23.7
pytest-cov==5.0.0
pytest-mock==3.14.0
pytest-randomly==3.10.1
python-dateutil==2.9.0.post0
pytz==2024.1
pyyaml==6.0.1
regex==2024.5.15
regex==2024.7.24
requests==2.32.3
s3transfer==0.10.2
sentencepiece==0.2.0
Expand All @@ -96,6 +97,6 @@ typing-extensions==4.12.2
typing-inspect==0.9.0
tzdata==2024.1
urllib3==2.2.2
vcrpy==6.0.1
vcrpy==5.1.0
wrapt==1.16.0
yarl==1.9.4
Loading
Loading