Understanding get(), chunk(), lazy(), and cursor() in Laravel

Introduction

When querying data, several factors can affect your application’s performance. Improper usage can lead to high memory consumption and significantly slow down page loading times.

In addition to using Laravel Octane to boost performance, Laravel provides several methods to query data more efficiently, depending on the use case. In this article, Laravel 11 and MySQL are used to demonstrate various examples.

Overview of Methods

• get(): Executes the query and loads all records into PHP memory.
• chunk(): Splits data into smaller groups and processes them sequentially.
• lazy(): Similar to chunk(), but uses PHP Generator to save more memory.
• cursor(): Retrieves records one by one directly from the database buffer for better memory efficiency.

get()

The get() method in Laravel executes the query and loads the entire result set into memory as a Collection.

• Executes the full query immediately.
• Not memory-efficient; all data is loaded at once.
• Supports Collection methods like map(), filter().
• Supports Eager Load Relationships.

⚠️ Note: Only use this method for queries that return small datasets.

🧪 Example: Retrieve all employees.

Laravel:

$employees = Employee::get();

Executed SQL:

SELECT * FROM employees;

Memory Usage:

With memory_limit = 128MB in php.ini, querying a table with 300,000 rows using get() results in the error “Allowed memory size of 134217728 bytes exhausted”.

In a real-world case, only about 70,000 rows could be retrieved with this dataset:

chunk()

The chunk() method allows you to process data in smaller sets (chunks) sequentially. It is suitable for handling large datasets, optimizing memory usage compared to loading everything at once.

• Ideal for large queries that can be processed in groups.
• Supports Collection methods like map(), filter().
• Supports Eager Load Relationships.

🧪 Example: Retrieve 500 employees per chunk.

Laravel:

Employee::chunk(500, function ($employees) {
    foreach ($employees as $employee) {
        echo $employee->first_name . "\n";
    }
});

Executed SQL:

Chunk 1:
SELECT * FROM `employees` ORDER BY `employees`.`emp_no` ASC LIMIT 500 OFFSET 0;
Chunk 2:
SELECT * FROM `employees` ORDER BY `employees`.`emp_no` ASC LIMIT 500 OFFSET 500;
Chunk 3:
SELECT * FROM `employees` ORDER BY `employees`.`emp_no` ASC LIMIT 500 OFFSET 1000;

Memory Usage:

For 300,000 rows with chunk size 500, the memory usage is around 6.07MB.

⚠️ Note: If filtering results based on a column while also updating it, you may encounter inconsistencies or skipped records due to shifting offsets.

🧪 Example: Update status from ‘accepted’ to ‘pending’ in chunks of 2.

Laravel:

Employee::where('status', 'accepted')->chunk(2, function ($employees) {
    foreach ($employees as $employee) {
        $employee->status = 'pending';
        $employee->save();
    }
});

Problem: Employees with emp_no: 10003, 10004, 10007, 10008 were skipped due to updates shifting the offset during execution.

Solution: Use chunkById() to avoid relying on OFFSET and instead filter based on the last processed ID.

Laravel:

Employee::where('status', 'accepted')->chunkById(2, function ($employees) {
    foreach ($employees as $employee) {
        $employee->status = 'pending';
        $employee->save();
    }
}, 'emp_no');

lazy()

The lazy() method processes records one-by-one using a LazyCollection, optimizing memory usage.

Unlike chunk(), lazy() doesn’t require a callback and internally behaves like cursor().

• Supports Collection methods like map(), filter().
• Supports Eager Load Relationships.
• Does not load all data into memory like get().

🧪 Example: Update status for employees with status = ‘accepted’.

Laravel:

$employees = Employee::where('status', 'accepted')->lazy(2);
foreach ($employees as $employee) {
    $employee->status = 'pending';
    $employee->save();
}

⚠️ Note: Like chunk(), lazy() can skip records when updating during iteration. To avoid this, use lazyById():

$employees = Employee::where('status', 'accepted')->lazyById(2, 'emp_no');
foreach ($employees as $employee) {
    $employee->status = 'pending';
    $employee->save();
}

cursor()

The cursor() method runs a single SQL query and yields records one by one directly from the database buffer. It doesn’t load the full result into PHP memory, making it efficient for large datasets.

• Best suited for processing large datasets.
• Does not support Collection methods like map(), filter().
• Does not support Eager Load Relationships.

🧪 Example: Print first names of all employees.

Laravel:

foreach (Employee::cursor() as $employee) {
    echo $employee->first_name . "\n";
}

Executed SQL:

SELECT * FROM employees;

Memory Usage:

With 300,000 rows, memory usage is around 1.87MB.

⚠️ Note:

Although cursor() is memory-efficient by loading one record at a time, it may still hit memory limits due to the underlying PDO driver behavior (buffering all results in memory).

In such cases, consider optimizing PDO settings or query logic, or using pagination, streaming, or job queue processing to prevent overloading a single process.

Summary

Huỳnh Hữu Phát Developer