Google Data Engineer : GCP-DE valid dump

You are running a pipeline in Cloud Dataflow that receives messages from a Cloud Pub/Sub topic and writes the results to a BigQuery dataset in the EU. Currently, your pipeline is located in europe-west4 and has a maximum of 3 workers, instance type n1-standard-1. You notice that during peak periods, your pipeline is struggling to process records in a timely fashion, when all 3 workers are at maximum CPU utilization. Which two actions can you take to increase performance of your pipeline? (Choose two.)

• A. Change the zone of your Cloud Dataflow pipeline to run in us-central1
• B. Create a temporary table in Cloud Bigtable that will act as a buffer for new data
• C. Use a larger instance type for your Cloud Dataflow workers
• D. Create a temporary table in Cloud Spanner that will act as a buffer for new data
• E. Create a new step in your pipeline to write to this table first, and then create a new pipeline to write from Cloud Spanner to BigQuery
• F. Increase the number of max workers
• G. Create a new step in your pipeline to write to this table first, and then create a new pipeline to write from Cloud Bigtable to BigQuery

You launched a new gaming app almost three years ago. You have been uploading log files from the previous day to a separate Google BigQuery table with the table name format LOGS_yyyymmdd. You have been using table wildcard functions to generate daily and monthly reports for all time ranges. Recently, you discovered that some queries that cover long date ranges are exceeding the limit of 1,000 tables and failing. How can you resolve this issue?

• A. Create separate views to cover each month, and query from these views
• B. Convert all daily log tables into date-partitioned tables
• C. Enable query caching so you can cache data from previous months
• D. Convert the sharded tables into a single partitioned table

What are two methods that can be used to denormalize tables in BigQuery?

• A. 1) Use a partitioned table; 2) Join tables into one table
• B. 1) Split table into multiple tables; 2) Use a partitioned table
• C. 1) Join tables into one table; 2) Use nested repeated fields
• D. 1) Use nested repeated fields; 2) Use a partitioned table

You are planning to migrate your current on-premises Apache Hadoop deployment to the cloud. You need to ensure that the deployment is as fault-tolerant and cost-effective as possible for long-running batch jobs. You want to use a managed service. What should you do?

• A. Change references in scripts from hdfs:// to gs://
• B. Store data in Cloud Storage, and change references in scripts from hdfs:// to gs://
• C. Install the Cloud Storage connector, and store the data in Cloud Storag
• D. Use an SSD persistent disk and 50% preemptible worker
• E. Install Hadoop and Spark on a 10-node Compute Engine instance group with standard instance
• F. Use a standard persistent disk and 50% preemptible worker
• G. Deploy a Cloud Dataproc cluste
• H. Store data in Cloud Storage, and change references in scripts from hdfs:// to gs://
• I. Install Hadoop and Spark on a 10-node Compute Engine instance group with preemptible instances.Store data in HDF
• J. Deploy a Cloud Dataproc cluste
• K. Change references in scripts from hdfs:// to gs://

You need to create a data pipeline that copies time-series transaction data so that it can be queried from within BigQuery by your data science team for analysis. Every hour, thousands of transactions are updated with a new status. The size of the intitial dataset is 1.5 PB, and it will grow by 3 TB per day. The data is heavily structured, and your data science team will build machine learning models based on this dat a. You want to maximize performance and usability for your data science team. Which two strategies should you adopt? Choose 2 answers.

• A. Copy a daily snapshot of transaction data to Cloud Storage and store it as an Avro fil
• B. Preserve the structure of the data as much as possible.
• C. Use BigQuery UPDATE to further reduce the size of the dataset.
• D. Develop a data pipeline where status updates are appended to BigQuery instead of updated.
• E. Use BigQuery'ssupport for external data sources to query.
• F. Denormalize the data as must as possible.