PySpark Certification Practice Test 02

DataFrame Operations and Transformations

Question 1 of 20

📋 Multiple Choice

Question 21

A newly joined Data Engineer at an e-commerce company receives a customer dataset containing more than 50 columns. Business users only need: - customer_id - customer_name - country The engineer wants to reduce unnecessary data processing as early as possible. Which PySpark operation should be used?

A. distinct()

B. filter()

C. union()

✓

D. select()

Answer

✅ D. select()

Explanation

### Explanation select() allows engineers to choose only required columns.

Answered: 0 / 20

PySpark Certification Practice Test 02

DataFrame Operations and Transformations

Insightful Saga — Modern Data Engineering Certification Preparation

Question 21

A. distinct()

B. filter()

C. union()

D. select()

Answer: ✅ D. select()

Explanation:

### Explanation select() allows engineers to choose only required columns.

Question 22

A retail company wants to identify transactions where: sales_amount > 10000 Which DataFrame operation is MOST appropriate?

A. filter()

B. select()

C. drop()

D. withColumn()

Answer: ✅ A. filter()

Question 23

A banking company stores customer information. Engineers want to create a new column: is_premium_customer based on account balance conditions. Which operation should be used?

A. withColumn()

B. select()

C. limit()

D. distinct()

Answer: ✅ A. withColumn()

Question 24

A healthcare company receives source files containing duplicate patient records. Engineers need unique patient rows before loading downstream systems. Which operation is most appropriate?

A. filter()

B. distinct()

C. orderBy()

D. cast()

Answer: ✅ B. distinct()

Question 25

A telecom company wants to sort customers from highest monthly usage to lowest. Which DataFrame operation should be applied?

A. orderBy()

B. agg()

C. filter()

D. drop()

Answer: ✅ A. orderBy()

Question 26

A newly joined Engineer notices a column: customer_temp_field is no longer needed. What is the BEST operation?

A. drop()

B. filter()

C. select()

D. cache()

Answer: ✅ A. drop()

Question 27

A retail company stores product prices as strings. Engineers must perform mathematical calculations. Which operation should be applied first?

A. cast()

B. distinct()

C. filter()

D. repartition()

Answer: ✅ A. cast()

Question 28

An insurance company wants to classify policyholders as: - High Risk - Medium Risk - Low Risk based on risk scores. Which PySpark function is most suitable?

A. union()

B. when()

C. filter()

D. drop()

Answer: ✅ B. when()

Question 29

A logistics company must return only shipments delivered successfully. Which operation should be applied?

A. filter()

B. union()

C. cast()

D. drop()

Answer: ✅ A. filter()

Question 30

A company receives customer data with: First_Name Business standards require: first_name What operation is MOST appropriate?

A. withColumnRenamed()

B. distinct()

C. count()

D. filter()

Answer: ✅ A. withColumnRenamed()

Question 31

A Data Engineer wants to know how many records exist after filtering. Which operation will trigger execution?

A. withColumn()

B. select()

C. count()

D. filter()

Answer: ✅ C. count()

Question 32

A retail company needs the first 100 records for testing. Which operation should be considered?

A. union()

B. limit()

C. cast()

D. repartition()

Answer: ✅ B. limit()

Question 33

Two regional sales datasets have identical schemas. Business users want both datasets combined into one. Which operation should be used?

A. drop()

B. cast()

C. distinct()

D. union()

Answer: ✅ D. union()

Question 34

A healthcare company needs the average patient age. Which operation category should be used?

A. Ordering

B. Aggregation

C. Casting

D. Filtering

Answer: ✅ B. Aggregation

Question 35

A banking company wants to determine the number of unique customers. What combination is MOST appropriate?

A. cast() + union()

B. drop() + limit()

C. distinct() + count()

D. filter() + show()

Answer: ✅ C. distinct() + count()

Question 36

A manufacturing company loads machine readings. Engineers create five new columns using withColumn(). When are these transformations actually executed?

A. During an Action

B. During Executor Startup

C. During DataFrame Creation

D. Immediately

Answer: ✅ A. During an Action

Explanation:

### Explanation PySpark transformations are lazily evaluated.

Question 37

A retail analyst requests all customer records where: Country = 'India' AND Sales > 50000 Which operation should be applied?

A. cast()

B. filter()

C. drop()

D. union()

Answer: ✅ B. filter()

Question 38

A telecom company wants to replace NULL values in a usage column with 0. Which operation is most appropriate?

A. fillna()

B. filter()

C. distinct()

D. orderBy()

Answer: ✅ A. fillna()

Question 39

A newly joined engineer notices some columns contain unexpected leading and trailing spaces. Which function should be evaluated?

A. collect()

B. trim()

C. repartition()

D. union()

Answer: ✅ B. trim()

Question 40

A global retailer processes billions of records daily. Management asks: "Why are operations like select(), filter(), and withColumn() called transformations?" Choose the BEST answer.

A. They increase cluster size

B. They trigger jobs immediately

C. They create executors

D. They define data processing logic without immediately executing it

Answer: ✅ D. They define data processing logic without immediately executing it

Explanation:

### Explanation Transformations build the logical execution plan. Spark executes them only when an action such as count(), show(), or collect() is triggered. --- --- title: PySpark Certification Practice Test 03 description: Data Engineer Review Notes --- # PySpark Certification Practice Test 03