PySpark Certification Practice Test 11

Spark Optimization Scenarios

Question 1 of 20

📋 Multiple Choice

Question 201

A PySpark job joins: - Sales Table = 4 TB - Product Lookup = 15 MB Spark UI shows: - Large shuffle - Long join stage Best optimization?

A. Collect()

✓

B. Broadcast Join

C. Cross Join

D. Coalesce(1)

Answer

✅ B. Broadcast Join

Explanation

### Explanation Broadcasting the small lookup table avoids a costly shuffle.

Answered: 0 / 20

PySpark Certification Practice Test 11

Spark Optimization Scenarios

Insightful Saga — Modern Data Engineering Certification Preparation

Question 201

A PySpark job joins: - Sales Table = 4 TB - Product Lookup = 15 MB Spark UI shows: - Large shuffle - Long join stage Best optimization?

A. Collect()

B. Broadcast Join

C. Cross Join

D. Coalesce(1)

Answer: ✅ B. Broadcast Join

Explanation:

### Explanation Broadcasting the small lookup table avoids a costly shuffle.

Question 202

Spark UI shows: - 200 Tasks - 199 finish quickly - 1 task runs for 45 minutes Most likely optimization?

A. Union

B. Salting for Data Skew

C. collect()

D. Cache

Answer: ✅ B

Question 203

A DataFrame is used in: - 5 joins - 3 aggregations - 4 transformations Spark recomputes it repeatedly. Best optimization?

A. Cache/Persist

B. count()

C. collect()

D. orderBy()

Answer: ✅ A

Question 204

A write operation creates: ```text 10,000 small files ``` Performance issues appear downstream. Best optimization?

A. Cross Join

B. Window Functions

C. Coalesce

D. collect()

Answer: ✅ C

Question 205

A job contains multiple joins. Spark 3.x is available. The team wants Spark to optimize join strategies automatically. Best feature?

A. Cache Removal

B. collect()

C. AQE (Adaptive Query Execution)

D. Union

Answer: ✅ C

Question 206

A table is partitioned by: ```text year month ``` Query: ```sql WHERE year = 2025 ``` Which optimization helps?

A. collect()

B. Cross Join

C. Partition Pruning

D. count()

Answer: ✅ C

Question 207

Spark UI shows: ```text Shuffle Read = Extremely High ``` during a join. What should be investigated first?

A. Variable Names

B. Notebook Title

C. Join Strategy

D. Comments

Answer: ✅ C

Question 208

A dataset contains: ```text 5 Billion Rows ``` Output needs only: ```text 100 Sample Records ``` Best approach?

A. limit(100)

B. orderBy()

C. collect()

D. count()

Answer: ✅ A

Question 209

A job repeatedly scans the same source files. Which optimization may help?

A. distinct()

B. collect()

C. Union

D. Cache

Answer: ✅ D

Question 210

A Data Engineer writes: ```python repartition(5000) ``` for a table containing only 100,000 rows. What should be reviewed?

A. NULL Handling

B. Partition Overhead

C. Window Functions

D. Delta Tables

Answer: ✅ B

Question 211

A Spark job performs: ```python orderBy() ``` on 2 billion rows. What major operation should be expected?

A. Cache

B. Broadcast

C. Global Shuffle

D. Partition Pruning

Answer: ✅ C

Question 212

A dimension table is used in nearly every ETL job. Its size: ```text 5 MB ``` Best optimization?

A. Broadcast

B. Full Join

C. Repartition(1)

D. Collect

Answer: ✅ A

Question 213

A Spark application experiences memory pressure. The cached dataset is rarely reused. Best action?

A. Remove Unnecessary Cache

B. Increase Actions

C. Add More collect()

D. More withColumn()

Answer: ✅ A

Question 214

Spark UI shows: ```text Thousands of tiny tasks ``` with minimal processing per task. Which area should be investigated?

A. Window Functions

B. UDF Logic

C. Broadcast Hints

D. Excessive Partition Count

Answer: ✅ D

Question 215

A job fails because one partition contains most of the data. Which issue is occurring?

A. Watermark Failure

B. Data Skew

C. Schema Drift

D. Cache Failure

Answer: ✅ B

Question 216

A company stores data in Delta format. Queries always filter by: ```text customer_id ``` What should engineers evaluate?

A. count()

B. union()

C. collect()

D. Z-Ordering

Answer: ✅ D

Question 217

Spark UI indicates: ```text Task Serialization Time ``` is unusually high. First review?

A. More Joins

B. Large Objects Sent to Executors

C. More Partitions

D. More Aggregations

Answer: ✅ B

Question 218

A DataFrame is used only once. A developer proposes: ```python cache() ``` Best recommendation?

A. Convert to RDD

B. Always persist

C. Caching may not provide value

D. Cache everything

Answer: ✅ C

Question 219

A Spark SQL query scans an entire table even though a filter is applied. What optimization should be verified?

A. Predicate Pushdown

B. Cross Join

C. count()

D. collect()

Answer: ✅ A

Question 220

A Senior Data Engineer is reviewing a slow PySpark job. Which area generally provides the largest performance gains?

A. Rearranging Notebook Cells

B. Adding Comments

C. Reducing Shuffles and Data Movement

D. Renaming Columns

Answer: ✅ C

Explanation:

### Explanation In large-scale Spark environments, performance improvements typically come from reducing shuffles, optimizing joins, handling skew, pruning partitions, and minimizing unnecessary data movement. --- --- title: PySpark Certification Practice Test 12 description: Principal Engineer and Architecture Scenarios --- # PySpark Certification Practice Test 12