Google Workspace
Google Workspace is a collection of cloud computing, productivity and collaboration tools, software and products developed and marketed by Google.
Logs
You can send your data to Logz.io using one of the following methods:
Google Workspace API
Extract data directly from the Google Workspace API and forward it to Logz.io using the Logz.io API Fetcher.
Prerequisites
Please follow Google guide to enable Google Workspace API under Before you begin section to configure the below:
Enable the APIs
Enable the following APIs in your Google Cloud project:
Create a service account
To allow Service-to-Service interactions to authenticate with the Google API, create a service account for your Google Cloud project.
Create a delegated user
Create a Super Admin user that impersonates the service account, and assign it to a new role which holds the privileges to the APIs you'd like to access.
Create a service account key
Create a Service account Key for the service account you created in step 2. Save the key JSON file in the same path where you'll save the API Fetcher configuration (and run the program from) later on.
Setup domain wide delegations
Setup Domain wide delegations so the Service account can access the APIs you'd like to access.
For Google Workspace Activity logs, the scope https://www.googleapis.com/auth/admin.reports.audit.readonly is enough.
Pull Docker Image
Download the logzio-api-fetcher image:
docker pull logzio/logzio-api-fetcher
Configuration
Create a local config file config.yaml. Save it in the same path where you saved your Google Workspace Service Account Key JSON.
apis:
- name: google saml
type: google_activity
google_ws_sa_file_name: credentials_file.json
google_ws_delegated_account: user@example.com
application_name: saml
additional_fields:
type: google_activity
days_back_fetch: 7
scrape_interval: 5
- name: google user accounts
type: google_activity
google_ws_sa_file_name: credentials_file.json
google_ws_delegated_account: user@example.com
application_name: user_accounts
additional_fields:
type: google_activity
days_back_fetch: 7
scrape_interval: 5
- name: google login
type: google_activity
google_ws_sa_file_name: credentials_file.json
google_ws_delegated_account: user@example.com
application_name: login
additional_fields:
type: google_activity
days_back_fetch: 7
scrape_interval: 5
- name: google admin
type: google_activity
google_ws_sa_file_name: credentials_file.json
google_ws_delegated_account: user@example.com
application_name: admin
additional_fields:
type: google_activity
days_back_fetch: 7
scrape_interval: 5
- name: google groups
type: google_activity
google_ws_sa_file_name: credentials_file.json
google_ws_delegated_account: user@example.com
application_name: groups
additional_fields:
type: google_activity
days_back_fetch: 7
scrape_interval: 5
logzio:
url: https://<<LISTENER-HOST>>:8071
token: <<LOG-SHIPPING-TOKEN>>
You can customize the endpoints to collect data by adding or modifying the configurations under the apis section. Refer to the relevant API documentation for more details.
Google Workspace Activity Configuration Options
To send Google Activity logs, use google_activity API type.
| Parameter Name | Description | Required/Optional | Default |
|---|---|---|---|
| name | Name of the API (custom name) | Optional | Google Workspace |
| google_ws_sa_file_name | The name of the service account credentials file. Required unless google_ws_sa_file_path is set. | Required* | "" |
| google_ws_sa_file_path | The path to the service account credentials file. Required unless google_ws_sa_file_name is set. Use this if mounting the file to a different path than the default. | Optional* | ./src/shared/<google_ws_sa_file_name> |
| google_ws_delegated_account | The email of the user for which the application is requesting delegated access | Required | - |
| application_name | Specifies the Google Workspace application to fetch activity data from (e.g., saml, user_accounts, login, admin, groups, etc). | Required | - |
| user_key | The unique ID of the user to fetch activity data for | Optional | all |
| additional_fields | Additional custom fields to add to the logs before sending to Logz.io | Optional | - |
| days_back_fetch | The amount of days to fetch back in the first request | Optional | 1 (day) |
| scrape_interval | Time interval to wait between runs (unit: minutes) | Optional | 1 (minute) |
Google Workspace General Configuration Options
To configure a different Google Workspace API as a source, use type as google_workspace, and configure it as necessary.
By default google_workspace API type has built in pagination settings and sets the response_data_path to items field.
| Parameter Name | Description | Required/Optional | Default |
|---|---|---|---|
| name | Name of the API (custom name) | Optional | Google Workspace |
| google_ws_sa_file_name | The name of the service account credentials file. Required unless google_ws_sa_file_path is set. | Required* | "" |
| google_ws_sa_file_path | The path to the service account credentials file. Required unless google_ws_sa_file_name is set. Use this if mounting the file to a different path than the default. | Optional* | ./src/shared/<google_ws_sa_file_name> |
| google_ws_delegated_account | The email of the user for which the application is requesting delegated access | Required | - |
| scopes | The OAuth 2.0 scopes that you might need to request to access Google APIs | Optional | ["https://www.googleapis.com/auth/admin.reports.audit.readonly"] |
| data_request | Nest here any detail relevant to the data request. (Options in General API) | Required | - |
| additional_fields | Additional custom fields to add to the logs before sending to Logz.io | Optional | - |
| days_back_fetch | The amount of days to fetch back in the first request | Optional | 1 (day) |
| scrape_interval | Time interval to wait between runs (unit: minutes) | Optional | 1 (minute) |
Logz.io output configuration options
To configure multiple outputs (in order to send different API data to different Logz.io accounts), please refer to the linked docs.
| Parameter Name | Description | Required/Optional | Default |
|---|---|---|---|
| url | The Logz.io Listener address | Optional | https://listener.logz.io:8071 |
| token | The Logz.io shipping token | Required | - |
Run The Docker Container
In the path where you saved your config.yaml and your Google Workspace Service account key JSON, run:
docker run --name logzio-api-fetcher \
-v "$(pwd)":/app/src/shared \
logzio/logzio-api-fetcher
To run in Debug mode add --level flag to the command:
docker run --name logzio-api-fetcher \
-v "$(pwd)":/app/src/shared \
logzio/logzio-api-fetcher \
--level DEBUG
Available Options: INFO, WARN, ERROR, DEBUG
Stopping the container
When you want to stop the container, to make sure it will finish the iteration on time, please give it a grace period of 30 seconds when you run the docker stop command:
docker stop -t 30 logzio-api-fetcher
Check Logz.io for your logs
Give your logs some time to get from your system to ours, and then open Open Search Dashboards.
Metrics
This integration is based on logzio-google-metrics.
Before you begin, you'll need:
- Login to your GCP account.
Run Google Cloud Shell configuration
Click this link to clone the solution's repo and use it in your Google Cloud Shell.
You may encounter a pop up window. Check the Trust repo checkbox, and press Confirm.
Run setup script in Google Cloud Shell
Copy the following snippet and paste in your Google Cloud Shell:
./run.sh --listener_url=<<LISTENER-HOST>> --token=<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> --gcp_region=<<GCP-REGION>> --function_name=<<FUNCTION-NAME-PREFIX>> --telemetry_list=<<TELEMETRY-LIST>>
When you run this script, you should choose the project ID where you need to run the integration.
Replace the variables as per the table below:
| Parameter | Description |
|---|---|
<<LISTENER-HOST>> | Use the listener URL specific to the region of your Logz.io account. You can look it up here. |
<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> | The metrics' shipping token of the account you want to ship to. |
<<GCP-REGION>> | Region where you want to upload Cloud Function. Requires for Deploy to Cloud option for platform. |
<<FUNCTION-NAME-PREFIX>> | Function name will be using as Google Cloud Function name. (Default: metrics_gcp) |
<<TELEMETRY-LIST>> | Will send metrics that match the Google metric type. Detailed list you can find here (ex: cloudfunctions.googleapis.com) |
Check Logz.io for your metrics
Give your data some time to get from your system to ours, then log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.