-
Notifications
You must be signed in to change notification settings - Fork 426
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merging master branch into sept_release branch (#1229)
* Updating issue template for adding issue SLO (#1216) * adding bug SLO * adding bug SLO * Fixing comment * Integration tests --key-file flag and GOOGLE_APPLICATION_CREDENTIALS env with admin permission tests (#1167) * updating go version * empty commit * local commit * local changes * local changes * local changes * adding key file tests * testing * testing * testing * testing * local changes * local changes * local changes * local changes * testing * testing * testing * testing * testing * adding test for admin creds * testing * testing * testing * testing * testing * testing * testing * testing * testing * testing * testing * testing * formating * testing defer statement * testing defer statement for deleting credentials * adding comment * testing with error * testing with error * testing with error * removing testing statement * adding testbucket and mntdir in commnd * adding comment * updating bucket name * updating bucket name * removing unnecessary changes * removing unnecessary changes * removing unnecessary changes * formatting * conflict * adding error handling * testing * small fix * removing creds tests from implicit and explicit dir tests * testing * testing * testing * testing * removing testing statement * adding creds tests in operations back * Testing * Testing * Testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * create service account key testing * adding remaining changes * adding remaining changes * adding remaining changes * testing service account * testing service account * testing service account * adding comments * removing unnecessary changes * formatting * testing * testing * testing * testing * removing without key file tests * small fix * formalizing for reuse * small fix * removing unnecessary changes * formatting * updating comment * updating comment * updating comment * fixing comments * adding comment * testing * testing * adding condintion for service account already exsit * adding condintion for service account already exsit * testing time * running tests only for operations * Changing the number of epoch based on previous observation. (#1222) * Adding rpm digest while creating rpm package (#1215) * removing defer as not working properly in for loop (#1223) --------- Co-authored-by: Tulsishah <[email protected]> Co-authored-by: Prince Kumar <[email protected]>
- Loading branch information
1 parent
23e5552
commit 6274ce8
Showing
11 changed files
with
196 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -111,5 +111,5 @@ | |
params=exp_config, | ||
model_dir=model_dir, | ||
run_post_eval=True, | ||
epochs=1000, | ||
epochs=675, | ||
clear_kernel_cache=True) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
// Copyright 2023 Google Inc. All Rights Reserved. | ||
// | ||
// Licensed under the Apache License, Version 2.0 (the "License"); | ||
// you may not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, software | ||
// distributed under the License is distributed on an "AS IS" BASIS, | ||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
// See the License for the specific language governing permissions and | ||
// limitations under the License. | ||
|
||
// Run tests for --key-file flag and GOOGLE_APPLICATION_CREDENTIALS env variable | ||
|
||
package creds_tests | ||
|
||
import ( | ||
"fmt" | ||
"log" | ||
"os" | ||
"path" | ||
"testing" | ||
|
||
"cloud.google.com/go/compute/metadata" | ||
"github.com/googlecloudplatform/gcsfuse/tools/integration_tests/util/mounting/static_mounting" | ||
"github.com/googlecloudplatform/gcsfuse/tools/integration_tests/util/setup" | ||
) | ||
|
||
const NameOfServiceAccount = "creds-test-gcsfuse" | ||
|
||
func setPermission(permission string, serviceAccount string) { | ||
// Provide permission to the bucket. | ||
setup.RunScriptForTestData("../util/creds_tests/testdata/provide_permission.sh", setup.TestBucket(), serviceAccount, permission) | ||
} | ||
|
||
func RunTestsForKeyFileAndGoogleApplicationCredentialsEnvVarSet(testFlagSet [][]string, permission string, m *testing.M) (successCode int) { | ||
// Fetching project-id to get service account id. | ||
id, err := metadata.ProjectID() | ||
if err != nil { | ||
log.Printf("Error in fetching project id: %v", err) | ||
} | ||
|
||
// Service account id format is [email protected] | ||
serviceAccount := NameOfServiceAccount + "@" + id + ".iam.gserviceaccount.com" | ||
|
||
// Create service account | ||
setup.RunScriptForTestData("../util/creds_tests/testdata/create_service_account.sh", NameOfServiceAccount, serviceAccount) | ||
|
||
key_file_path := path.Join(os.Getenv("HOME"), "creds.json") | ||
|
||
// Create credentials | ||
setup.RunScriptForTestData("../util/creds_tests/testdata/create_key_file.sh", key_file_path, serviceAccount) | ||
|
||
// Provide permission to service account for testing. | ||
setPermission(permission, serviceAccount) | ||
|
||
// Revoke the permission and delete creds and service account after testing. | ||
defer setup.RunScriptForTestData("../util/creds_tests/testdata/revoke_permission_and_delete_service_account_and_creds.sh", serviceAccount, key_file_path) | ||
|
||
// Without –key-file flag and GOOGLE_APPLICATION_CREDENTIALS | ||
// This case will not get covered as gcsfuse internally authenticates from a metadata server on GCE VM. | ||
// https://github.com/golang/oauth2/blob/master/google/default.go#L160 | ||
|
||
// Testing with GOOGLE_APPLICATION_CREDENTIALS env variable | ||
err = os.Setenv("GOOGLE_APPLICATION_CREDENTIALS", key_file_path) | ||
if err != nil { | ||
setup.LogAndExit(fmt.Sprintf("Error in setting environment variable: %v", err)) | ||
} | ||
|
||
successCode = static_mounting.RunTests(testFlagSet, m) | ||
|
||
if successCode != 0 { | ||
return | ||
} | ||
|
||
// Testing with --key-file and GOOGLE_APPLICATION_CREDENTIALS env variable set | ||
keyFileFlag := "--key-file=" + key_file_path | ||
|
||
for i := 0; i < len(testFlagSet); i++ { | ||
testFlagSet[i] = append(testFlagSet[i], keyFileFlag) | ||
} | ||
|
||
successCode = static_mounting.RunTests(testFlagSet, m) | ||
|
||
if successCode != 0 { | ||
return | ||
} | ||
|
||
err = os.Unsetenv("GOOGLE_APPLICATION_CREDENTIALS") | ||
if err != nil { | ||
setup.LogAndExit(fmt.Sprintf("Error in unsetting environment variable: %v", err)) | ||
} | ||
|
||
// Testing with --key-file flag only | ||
successCode = static_mounting.RunTests(testFlagSet, m) | ||
|
||
if successCode != 0 { | ||
return | ||
} | ||
|
||
return successCode | ||
} |
17 changes: 17 additions & 0 deletions
17
tools/integration_tests/util/creds_tests/testdata/create_key_file.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
# Copyright 2023 Google Inc. All Rights Reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http:#www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
KEY_FILE_PATH=$1 | ||
SERVICE_ACCOUNT=$2 | ||
gcloud iam service-accounts keys create $KEY_FILE_PATH --iam-account=$SERVICE_ACCOUNT |
22 changes: 22 additions & 0 deletions
22
tools/integration_tests/util/creds_tests/testdata/create_service_account.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# Copyright 2023 Google Inc. All Rights Reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http:#www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
SERVICE_ACCOUNT=$1 | ||
SERVICE_ACCOUNT_ID=$2 | ||
# Delete service account if already exist. | ||
gcloud iam service-accounts delete $SERVICE_ACCOUNT_ID | ||
if [ $? -eq 1 ]; then | ||
echo "Service account does not exist." | ||
fi | ||
gcloud iam service-accounts create $SERVICE_ACCOUNT --description="$SERVICE_ACCOUNT" --display-name="$SERVICE_ACCOUNT" |
20 changes: 20 additions & 0 deletions
20
tools/integration_tests/util/creds_tests/testdata/provide_permission.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Copyright 2023 Google Inc. All Rights Reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http:#www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
# Provide permission to the bucket. | ||
TEST_BUCKET=$1 | ||
SERVICE_ACCOUNT=$2 | ||
PERMISSION=$3 | ||
|
||
gsutil iam ch serviceAccount:$SERVICE_ACCOUNT:$PERMISSION gs://$TEST_BUCKET |
20 changes: 20 additions & 0 deletions
20
...tests/util/creds_tests/testdata/revoke_permission_and_delete_service_account_and_creds.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Copyright 2023 Google Inc. All Rights Reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http:#www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
# Delete service account after testing | ||
SERVICE_ACCOUNT=$1 | ||
KEY_FILE=$2 | ||
gcloud auth revoke $SERVICE_ACCOUNT | ||
gcloud iam service-accounts delete $SERVICE_ACCOUNT | ||
rm $KEY_FILE |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters