GCP Workspace

Google Workspace is a collection of cloud computing, productivity and collaboration tools, software and products developed and marketed by Google.

Logs

You can send your data to Logz.io using one of the following methods:

Google Pub\Sub

Google Cloud Pub/Sub

Forward Google Workspace Logs to Google Cloud Platform (GCP) and use Logz.io’s Pub/Sub integration to forward the data to Logz.io.

note

This integration is based on logzio-google-pubsub.

Before you begin, you'll need:

• Login to your GCP account.

Run Google Cloud Shell configuration

Click this link to clone the solution's repo and use it in your Google Cloud Shell.

note

If a pop-up window appears, check the Trust repo box and press Confirm.

Run setup script in Google Cloud Shell

Copy the following snippet and paste in your Google Cloud Shell:

./run.sh --listener_url=<<LISTENER-HOST>> --token=<<LOG-SHIPPING-TOKEN>> --gcp_region=<<GCP-REGION>> --log_type=<<LOG-TYPE>> --function_name=<<FUNCTION-NAME>> --telemetry_list=<<TELEMETRY-LIST>>

When you run this script, you should choose the project ID where you need to run the integration.

Replace the variables as per the table below:

Parameter	Description
<<LISTENER-HOST>>	Use the listener URL specific to the region of your Logz.io account. You can look it up here.
<<LOG-SHIPPING-TOKEN>>	The logs' shipping token of the account you want to ship to.
<<GCP-REGION>>	Region where you want to upload Cloud Function. Requires for Deploy to Cloud option for platform.
<<LOG-TYPE>>	Log type. Help classify logs into different classifications. (Default: gcp-pubsub)
<<FUNCTION-NAME>>	Function name will be using as Google Cloud Function name. (Default: logzioHandler)
<<TELEMETRY-LIST>>	Will send logs that match the Google resource type. Detailed list you can find here (ex: pubsub_topic,pubsub_subscription). For all services insert all_services.

Updating telemetry_list after creation

To update the resources that are monitored by the function follow the steps:

1. Go to Log router page.
2. Choose logzioHandler-sink-logs-to-logzio.
3. Edit the sink.
4. Update the query which filters for the resource types to monitor.

Google API

Google Workspace API

Extract data directly from the Google Workspace API and forward it to Logz.io using the Logz.io API Fetcher.

Prerequisites

Please follow Google guide to enable Google Workspace API under Before you begin section to configure the below:

Enable the APIs

Enable the following APIs in your Google Cloud project:

• Google Workspace Admin SDK
• Alert Center API

Create a service account

To allow Service-to-Service interactions to authenticate with the Google API, create a service account for your Google Cloud project.

Create a delegated user

Create a Super Admin user that impersonates the service account, and assign it to a new role which holds the privileges to the APIs you'd like to access.

Create a service account key

Create a Service account Key for the service account you created in step 2. Save the key JSON file in the same path where you'll save the API Fetcher configuration (and run the program from) later on.

Setup domain wide delegations

Setup Domain wide delegations so the Service account can access the APIs you'd like to access.

note

For Google Workspace Activity logs, the scope https://www.googleapis.com/auth/admin.reports.audit.readonly is enough.

Pull Docker Image

Download the logzio-api-fetcher image:

docker pull logzio/logzio-api-fetcher

Configuration

Create a local config file config.yaml. Save it in the same path where you saved your Google Workspace Service Account Key JSON.

apis:
  - name: google saml
    type: google_activity
    google_ws_sa_file_name: credentials_file.json
    google_ws_delegated_account: user@example.com
    application_name: saml
    additional_fields:
      type: google_activity
    days_back_fetch: 7
    scrape_interval: 5

  - name: google user accounts
    type: google_activity
    google_ws_sa_file_name: credentials_file.json
    google_ws_delegated_account: user@example.com
    application_name: user_accounts
    additional_fields:
      type: google_activity
    days_back_fetch: 7
    scrape_interval: 5

  - name: google login
    type: google_activity
    google_ws_sa_file_name: credentials_file.json
    google_ws_delegated_account: user@example.com
    application_name: login
    additional_fields:
      type: google_activity
    days_back_fetch: 7
    scrape_interval: 5

  - name: google admin
    type: google_activity
    google_ws_sa_file_name: credentials_file.json
    google_ws_delegated_account: user@example.com
    application_name: admin
    additional_fields:
      type: google_activity
    days_back_fetch: 7
    scrape_interval: 5

  - name: google groups
    type: google_activity
    google_ws_sa_file_name: credentials_file.json
    google_ws_delegated_account: user@example.com
    application_name: groups
    additional_fields:
      type: google_activity
    days_back_fetch: 7
    scrape_interval: 5

logzio:
  url: https://<<LISTENER-HOST>>:8071
  token: <<LOG-SHIPPING-TOKEN>>

note

You can customize the endpoints to collect data by adding or modifying the configurations under the apis section. Refer to the relevant API documentation for more details.

Google Workspace Activity Configuration Options

To send Google Activity logs, use google_activity API type.

Parameter Name	Description	Required/Optional	Default
name	Name of the API (custom name)	Optional	Google Workspace
google_ws_sa_file_name	The name of the service account credentials file. Required unless google_ws_sa_file_path is set.	Required*	""
google_ws_sa_file_path	The path to the service account credentials file. Required unless google_ws_sa_file_name is set. Use this if mounting the file to a different path than the default.	Optional*	./src/shared/<google_ws_sa_file_name>
google_ws_delegated_account	The email of the user for which the application is requesting delegated access	Required	-
application_name	Specifies the Google Workspace application to fetch activity data from (e.g., saml, user_accounts, login, admin, groups, etc).	Required	-
user_key	The unique ID of the user to fetch activity data for	Optional	all
additional_fields	Additional custom fields to add to the logs before sending to Logz.io	Optional	-
days_back_fetch	The amount of days to fetch back in the first request	Optional	1 (day)
scrape_interval	Time interval to wait between runs (unit: minutes)	Optional	1 (minute)

Google Workspace General Configuration Options

To configure a different Google Workspace API as a source, use type as google_workspace, and configure it as necessary.

By default google_workspace API type has built in pagination settings and sets the response_data_path to items field.

Parameter Name	Description	Required/Optional	Default
name	Name of the API (custom name)	Optional	Google Workspace
google_ws_sa_file_name	The name of the service account credentials file. Required unless google_ws_sa_file_path is set.	Required*	""
google_ws_sa_file_path	The path to the service account credentials file. Required unless google_ws_sa_file_name is set. Use this if mounting the file to a different path than the default.	Optional*	./src/shared/<google_ws_sa_file_name>
google_ws_delegated_account	The email of the user for which the application is requesting delegated access	Required	-
scopes	The OAuth 2.0 scopes that you might need to request to access Google APIs	Optional	["https://www.googleapis.com/auth/admin.reports.audit.readonly"]
data_request	Nest here any detail relevant to the data request. (Options in General API)	Required	-
additional_fields	Additional custom fields to add to the logs before sending to Logz.io	Optional	-
days_back_fetch	The amount of days to fetch back in the first request	Optional	1 (day)
scrape_interval	Time interval to wait between runs (unit: minutes)	Optional	1 (minute)

Logz.io output configuration options

note

To configure multiple outputs (in order to send different API data to different Logz.io accounts), please refer to the linked docs.

Parameter Name	Description	Required/Optional	Default
url	The Logz.io Listener address	Optional	https://listener.logz.io:8071
token	The Logz.io shipping token	Required	-

Run The Docker Container

In the path where you saved your config.yaml and your Google Workspace Service account key JSON, run:

docker run --name logzio-api-fetcher \
-v "$(pwd)":/app/src/shared \
logzio/logzio-api-fetcher 

note

To run in Debug mode add --level flag to the command:

docker run --name logzio-api-fetcher \
-v "$(pwd)":/app/src/shared \
logzio/logzio-api-fetcher \
--level DEBUG

Available Options: INFO, WARN, ERROR, DEBUG

Stopping the container

When you want to stop the container, to make sure it will finish the iteration on time, please give it a grace period of 30 seconds when you run the docker stop command:

docker stop -t 30 logzio-api-fetcher

Check Logz.io for your logs

Give your logs some time to get from your system to ours, and then open Open Search Dashboards.

Metrics

note

This integration is based on logzio-google-metrics.

Before you begin, you'll need:

• Login to your GCP account.

Run Google Cloud Shell configuration

Click this link to clone the solution's repo and use it in your Google Cloud Shell.

note

You may encounter a pop up window. Check the Trust repo checkbox, and press Confirm.

Run setup script in Google Cloud Shell

Copy the following snippet and paste in your Google Cloud Shell:

./run.sh --listener_url=<<LISTENER-HOST>> --token=<<PROMETHEUS-METRICS-SHIPPING-TOKEN>> --gcp_region=<<GCP-REGION>> --function_name=<<FUNCTION-NAME-PREFIX>> --telemetry_list=<<TELEMETRY-LIST>>

When you run this script, you should choose the project ID where you need to run the integration.

Replace the variables as per the table below:

Parameter	Description
<<LISTENER-HOST>>	Use the listener URL specific to the region of your Logz.io account. You can look it up here.
<<PROMETHEUS-METRICS-SHIPPING-TOKEN>>	The metrics' shipping token of the account you want to ship to.
<<GCP-REGION>>	Region where you want to upload Cloud Function. Requires for Deploy to Cloud option for platform.
<<FUNCTION-NAME-PREFIX>>	Function name will be using as Google Cloud Function name. (Default: metrics_gcp)
<<TELEMETRY-LIST>>	Will send metrics that match the Google metric type. Detailed list you can find here (ex: cloudfunctions.googleapis.com)

Check Logz.io for your metrics

Give your data some time to get from your system to ours, then log in to your Logz.io Metrics account, and open the Logz.io Metrics tab.