Skip to content

Commit 7088e39

Browse files
daniel-sanchegcf-owl-bot[bot]
daniel-sanche
and
gcf-owl-bot[bot]
authored
chore: add experimental async data client (#920)
* feat: add new v3.0.0 API skeleton (#745) * feat: improve rows filters (#751) * feat: read rows query model class (#752) * feat: implement row and cell model classes (#753) * feat: add pooled grpc transport (#748) * feat: implement read_rows (#762) * feat: implement mutate rows (#769) * feat: literal value filter (#767) * feat: row_exists and read_row (#778) * feat: read_modify_write and check_and_mutate_row (#780) * feat: sharded read rows (#766) * feat: ping and warm with metadata (#810) * feat: mutate rows batching (#770) * chore: restructure module paths (#816) * feat: improve timeout structure (#819) * fix: api errors apply to all bulk mutations * chore: reduce public api surface (#820) * feat: improve error group tracebacks on < py11 (#825) * feat: optimize read_rows (#852) * chore: add user agent suffix (#842) * feat: optimize retries (#854) * feat: add test proxy (#836) * chore(tests): add conformance tests to CI for v3 (#870) * chore(tests): turn off fast fail for conformance tets (#882) * feat: add TABLE_DEFAULTS enum for table method arguments (#880) * fix: pass None for retry in gapic calls (#881) * feat: replace internal dictionaries with protos in gapic calls (#875) * chore: optimize gapic calls (#863) * feat: expose retryable error codes to users (#879) * chore: update api_core submodule (#897) * chore: merge main into experimental_v3 (#900) * chore: pin conformance tests to v0.0.2 (#903) * fix: bulk mutation eventual success (#909) --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
1 parent 3ae3599 commit 7088e39

File tree

93 files changed

+21353
-277
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

93 files changed

+21353
-277
lines changed

‎.coveragerc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ omit =
2222
google/cloud/bigtable_admin/gapic_version.py
2323

2424
[report]
25-
fail_under = 100
25+
fail_under = 99
2626
show_missing = True
2727
exclude_lines =
2828
# Re-enable the standard pragma

‎.github/sync-repo-settings.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,24 @@ branchProtectionRules:
3131
- 'Kokoro'
3232
- 'Kokoro system-3.8'
3333
- 'cla/google'
34+
- pattern: experimental_v3
35+
# Can admins overwrite branch protection.
36+
# Defaults to `true`
37+
isAdminEnforced: false
38+
# Number of approving reviews required to update matching branches.
39+
# Defaults to `1`
40+
requiredApprovingReviewCount: 1
41+
# Are reviews from code owners required to update matching branches.
42+
# Defaults to `false`
43+
requiresCodeOwnerReviews: false
44+
# Require up to date branches
45+
requiresStrictStatusChecks: false
46+
# List of required status check contexts that must pass for commits to be accepted to matching branches.
47+
requiredStatusCheckContexts:
48+
- 'Kokoro'
49+
- 'Kokoro system-3.8'
50+
- 'cla/google'
51+
- 'Conformance / Async v3 Client / Python 3.8'
3452
# List of explicit permissions to add (additive only)
3553
permissionRules:
3654
# Team slug to add to repository permissions

‎.github/workflows/conformance.yaml‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Copyright 2023 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
# Github action job to test core java library features on
15+
# downstream client libraries before they are released.
16+
on:
17+
push:
18+
branches:
19+
- main
20+
pull_request:
21+
name: Conformance
22+
jobs:
23+
conformance:
24+
runs-on: ubuntu-latest
25+
strategy:
26+
matrix:
27+
test-version: [ "v0.0.2" ]
28+
py-version: [ 3.8 ]
29+
client-type: [ "Async v3", "Legacy" ]
30+
fail-fast: false
31+
name: "${{ matrix.client-type }} Client / Python ${{ matrix.py-version }} / Test Tag ${{ matrix.test-version }}"
32+
steps:
33+
- uses: actions/checkout@v3
34+
name: "Checkout python-bigtable"
35+
- uses: actions/checkout@v3
36+
name: "Checkout conformance tests"
37+
with:
38+
repository: googleapis/cloud-bigtable-clients-test
39+
ref: ${{ matrix.test-version }}
40+
path: cloud-bigtable-clients-test
41+
- uses: actions/setup-python@v4
42+
with:
43+
python-version: ${{ matrix.py-version }}
44+
- uses: actions/setup-go@v4
45+
with:
46+
go-version: '>=1.20.2'
47+
- run: chmod +x .kokoro/conformance.sh
48+
- run: pip install -e .
49+
name: "Install python-bigtable from HEAD"
50+
- run: go version
51+
- run: .kokoro/conformance.sh
52+
name: "Run tests"
53+
env:
54+
CLIENT_TYPE: ${{ matrix.client-type }}
55+
PYTHONUNBUFFERED: 1
56+

‎.github/workflows/system_emulated.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ jobs:
2020
python-version: '3.8'
2121

2222
- name: Setup GCloud SDK
23-
uses: google-github-actions/setup-gcloud@v2.0.1
23+
uses: google-github-actions/setup-gcloud@v2.0.0
2424

2525
- name: Install / run Nox
2626
run: |

‎.github/workflows/unittest.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,4 +54,4 @@ jobs:
5454
run: |
5555
find .coverage-results -type f -name '*.zip' -exec unzip {} \;
5656
coverage combine .coverage-results/**/.coverage*
57-
coverage report --show-missing --fail-under=100
57+
coverage report --show-missing --fail-under=99

‎.gitmodules‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
[submodule "python-api-core"]
2+
path = python-api-core
3+
url = git@github.com:googleapis/python-api-core.git
4+
[submodule "gapic-generator-fork"]
5+
path = gapic-generator-fork
6+
url = git@github.com:googleapis/gapic-generator-python.git

‎.kokoro/conformance.sh‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
#!/bin/bash
2+
3+
# Copyright 2023 Google LLC
4+
#
5+
# Licensed under the Apache License, Version 2.0 (the "License");
6+
# you may not use this file except in compliance with the License.
7+
# You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
17+
set -eo pipefail
18+
19+
## cd to the parent directory, i.e. the root of the git repo
20+
cd $(dirname $0)/..
21+
22+
PROXY_ARGS=""
23+
TEST_ARGS=""
24+
if [[ "${CLIENT_TYPE^^}" == "LEGACY" ]]; then
25+
echo "Using legacy client"
26+
PROXY_ARGS="--legacy-client"
27+
# legacy client does not expose mutate_row. Disable those tests
28+
TEST_ARGS="-skip TestMutateRow_"
29+
fi
30+
31+
# Build and start the proxy in a separate process
32+
PROXY_PORT=9999
33+
pushd test_proxy
34+
nohup python test_proxy.py --port $PROXY_PORT $PROXY_ARGS &
35+
proxyPID=$!
36+
popd
37+
38+
# Kill proxy on exit
39+
function cleanup() {
40+
echo "Cleanup testbench";
41+
kill $proxyPID
42+
}
43+
trap cleanup EXIT
44+
45+
# Run the conformance test
46+
pushd cloud-bigtable-clients-test/tests
47+
eval "go test -v -proxy_addr=:$PROXY_PORT $TEST_ARGS"
48+
RETURN_CODE=$?
49+
popd
50+
51+
echo "exiting with ${RETURN_CODE}"
52+
exit ${RETURN_CODE}

‎.kokoro/presubmit/conformance.cfg‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Format: //devtools/kokoro/config/proto/build.proto
2+
3+
env_vars: {
4+
key: "NOX_SESSION"
5+
value: "conformance"
6+
}

‎gapic-generator-fork‎

Submodule gapic-generator-fork added at b26cda7

‎google/cloud/bigtable/data/__init__.py‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
# -*- coding: utf-8 -*-
2+
# Copyright 2023 Google LLC
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# http://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
#
16+
from google.cloud.bigtable import gapic_version as package_version
17+
18+
from google.cloud.bigtable.data._async.client import BigtableDataClientAsync
19+
from google.cloud.bigtable.data._async.client import TableAsync
20+
21+
from google.cloud.bigtable.data._async.mutations_batcher import MutationsBatcherAsync
22+
23+
from google.cloud.bigtable.data.read_rows_query import ReadRowsQuery
24+
from google.cloud.bigtable.data.read_rows_query import RowRange
25+
from google.cloud.bigtable.data.row import Row
26+
from google.cloud.bigtable.data.row import Cell
27+
28+
from google.cloud.bigtable.data.mutations import Mutation
29+
from google.cloud.bigtable.data.mutations import RowMutationEntry
30+
from google.cloud.bigtable.data.mutations import SetCell
31+
from google.cloud.bigtable.data.mutations import DeleteRangeFromColumn
32+
from google.cloud.bigtable.data.mutations import DeleteAllFromFamily
33+
from google.cloud.bigtable.data.mutations import DeleteAllFromRow
34+
35+
from google.cloud.bigtable.data.exceptions import InvalidChunk
36+
from google.cloud.bigtable.data.exceptions import FailedMutationEntryError
37+
from google.cloud.bigtable.data.exceptions import FailedQueryShardError
38+
39+
from google.cloud.bigtable.data.exceptions import RetryExceptionGroup
40+
from google.cloud.bigtable.data.exceptions import MutationsExceptionGroup
41+
from google.cloud.bigtable.data.exceptions import ShardedReadRowsExceptionGroup
42+
43+
from google.cloud.bigtable.data._helpers import TABLE_DEFAULT
44+
from google.cloud.bigtable.data._helpers import RowKeySamples
45+
from google.cloud.bigtable.data._helpers import ShardedQuery
46+
47+
48+
__version__: str = package_version.__version__
49+
50+
__all__ = (
51+
"BigtableDataClientAsync",
52+
"TableAsync",
53+
"RowKeySamples",
54+
"ReadRowsQuery",
55+
"RowRange",
56+
"MutationsBatcherAsync",
57+
"Mutation",
58+
"RowMutationEntry",
59+
"SetCell",
60+
"DeleteRangeFromColumn",
61+
"DeleteAllFromFamily",
62+
"DeleteAllFromRow",
63+
"Row",
64+
"Cell",
65+
"InvalidChunk",
66+
"FailedMutationEntryError",
67+
"FailedQueryShardError",
68+
"RetryExceptionGroup",
69+
"MutationsExceptionGroup",
70+
"ShardedReadRowsExceptionGroup",
71+
"ShardedQuery",
72+
"TABLE_DEFAULT",
73+
)

0 commit comments

Comments
 (0)