In this short article, we will examine some non-obvious performance issues that can arise when using std::vector<char> as an output buffer for input-output operations. Specifically, we will discuss the consequences of using the resize and reserve methods and how their improper use can lead to undesirable outcomes.
As we know, std::vector is a dynamic array that provides convenient management of a contiguous block of memory. It is not surprising that it is a popular choice for input-output operations, especially when working with data of varying sizes. However, when using std::vector<char> as an output buffer, it is important to efficiently allocate and manage memory to avoid unexpected performance consequences.
One common mistake is using the resize method instead of the reserve method to allocate memory for the output buffer. Although both methods can allocate memory, they serve different purposes:
reserve: This method increases the capacity of the vector without changing its size or initializing new elements. It is useful when you want to pre-allocate memory to avoid repeated memory reallocations during input-output operations.resize: This method changes the size of the vector, initializing new elements if necessary. Although it can allocate memory, it also has a side effect of initializing new elements, which can lead to unnecessary overhead, especially when working with large buffers.
In particular, we will examine the performance issue that can arise when using the resize method to allocate memory for the output buffer, as it not only allocates memory but also initializes elements. This initialization process can consume a significant amount of CPU resources and have a substantial impact on performance.
Let’s assume we have the following C++ class responsible for reading data from a driver, and the io_handler::read_data function is frequently called from the main code, reading arbitrary amounts of bytes that can vary significantly from operation to operation. Please note that this example is far from real code and has been greatly simplified for the purposes of this article.
class io_handler
{
static constexpr size_t input_buffer_size = 10 * 1024; // 10KB
static constexpr size_t output_buffer_size = 10 * 1024 * 1024; // 10MB
HANDLE file_handle_{ nullptr };
std::vector<char> input_buffer_;
std::vector<char> output_buffer_;
public:
io_handler()
{
input_buffer_.reserve(input_buffer_size); // Reserve space in the input buffer
output_buffer_.reserve(output_buffer_size); // Reserve space in the output buffer
// TODO: Create the file handle using ::CreateFile() function
}
~io_handler()
{
CloseHandle(file_handle_); // Close the file handle
}
io_handler(const io_handler& other) = delete;
io_handler(io_handler&& other) = delete;
io_handler& operator=(const io_handler& other) = delete;
io_handler& operator=(io_handler&& other) = delete;
// Read data from the input buffer and return true if successful, false otherwise
[[nodiscard]] bool read_data()
{
// Resize the output buffer to maximum size
output_buffer_.resize(output_buffer_.capacity());
DWORD out_bytes = 0;
const BOOL result = ::DeviceIoControl(
file_handle_, // File handle to use
IOCTL_READ_DATA_CHUNK, // IOCTL code
input_buffer_.data(), // Input buffer data
static_cast<DWORD>(input_buffer_.size()), // Input buffer size
output_buffer_.data(), // Output buffer data
static_cast<DWORD>(output_buffer_.capacity()), // Output buffer size
&out_bytes, // Number of bytes received
nullptr // No overlapped structure
);
// Resize the vector to the actual number of bytes received
output_buffer_.resize(out_bytes);
return result;
}
};On the one hand, the given code works as expected, and at first glance, it seems that there are no problems. However, the line output_buffer_.resize(output_buffer_.capacity()) shows significant CPU resource consumption during profiling, even surpassing the call to DeviceIoControl. What could be the problem?
Indeed, changing the size of a vector involves allocating new memory and moving existing elements, which in itself is quite an expensive operation. However, output_buffer_.resize(output_buffer_.capacity()) should not cause vector reallocation. Nonetheless, as mentioned earlier, the resize method does more than just allocate memory; it also initializes the elements with their default values, and if the vector is large enough, as in our case, this process can consume a significant amount of CPU resources.
To prevent the initialization of the char type, you can wrap it in a class or structure with an empty default constructor. For example, consider the no_init_primitive_type template class, which serves as a wrapper around primitive types and disables unnecessary default initialization for them. By using this class, you can create objects with uninitialized values, which can be useful in situations where initialization is not required.
/**
* @brief A template class for creating wrapper objects around primitive
* types without default initialization.
*
* This template class restricts the type `T` to be a primitive type except
* `void`, and disables default initialization of the wrapped value.
*
* @tparam T The type of the wrapped value.
*/
template <typename T,
typename = std::enable_if_t<std::is_trivial_v<T> && !std::is_void_v<T>>>
class no_init_primitive_type {
public:
/**
* @brief Constructs a new `no_init_primitive_type` object.
*
* This constructor creates a new `no_init_primitive_type` object
* with uninitialized wrapped value.
* Static assertions are used to ensure that the alignment and size of the
* type `T` match the alignment and size of the `no_init_primitive_type` class.
*/
no_init_primitive_type() {
static_assert(alignof(T) == alignof(no_init_primitive_type),
"Alignment of no_init_primitive_type does not match type alignment");
static_assert(sizeof(T) == sizeof(no_init_primitive_type),
"Size of no_init_primitive_type does not match type size");
}
/**
* @brief The wrapped value.
*/
T value;
};Here’s a modernized version of the class template using C++20 concepts:
#include <concepts>
/**
* @brief A template class for creating wrapper objects around primitive
* types without default initialization.
*
* This template class restricts the type `T` to be a primitive type except
* `void`, and disables default initialization of the wrapped value.
*
* @tparam T The type of the wrapped value.
*/
template <typename T>
requires std::is_trivial_v<T> && !std::is_void_v<T>
class no_init_primitive_type {
public:
/**
* @brief Constructs a new `no_init_primitive_type` object.
*
* This constructor creates a new `no_init_primitive_type` object
* with uninitialized wrapped value.
* Static assertions are used to ensure that the alignment and size of the
* type `T` match the alignment and size of the `no_init_primitive_type` class.
*/
no_init_primitive_type() {
static_assert(alignof(T) == alignof(no_init_primitive_type),
"Alignment of no_init_primitive_type does not match type alignment");
static_assert(sizeof(T) == sizeof(no_init_primitive_type),
"Size of no_init_primitive_type does not match type size");
}
/**
* @brief The wrapped value.
*/
T value;
};
This updated version uses C++20 concepts to simplify the constraints on the template parameter T. The requires keyword is used to apply the constraints directly in the class declaration, making the code more readable and concise.
Utilizing this template, we can revise the read_data() code as shown below:
// Read data from the input buffer and return true if successful, false otherwise
[[nodiscard]] bool read_data()
{
// By default, resizing a vector initializes the new elements
// to their default values, which can significantly impact performance
// when dealing with large vectors. To avoid this, we omit the resize
// and pass the entire vector to the IoControl function.
// After the function returns, we resize the vector to the actual number
// of bytes received.
// This avoids default initialization and improves performance.
// output_buffer_.resize(output_buffer_.capacity());
DWORD out_bytes = 0;
const BOOL result = ::DeviceIoControl(
file_handle_, // File handle to use
IOCTL_READ_DATA_CHUNK, // IOCTL code
input_buffer_.data(), // Input buffer data
static_cast<DWORD>(input_buffer_.size()), // Input buffer size
output_buffer_.data(), // Output buffer data
static_cast<DWORD>(output_buffer_.capacity()), // Output buffer size
&out_bytes, // Number of bytes received
nullptr // No overlapped structure
);
// Resize the vector to the actual number of bytes received
reinterpret_cast<std::vector<no_init_primitive_type<char>>&>(output_buffer_)
.resize(out_bytes);
return result;
}This solution does not claim to be elegant or absolutely correct, but it does allow for addressing performance issues with minimal intervention in the existing code.
Another approach to address the inefficiencies associated with unnecessary initialization of vector elements is to provide a custom Allocator::construct. This method is detailed in a Stack Overflow discussion, where a solution involving the use of a custom allocator is proposed. This allocator, named default_init_allocator, interposes on the construct() calls to convert value initialization into default initialization.
The key advantage of this approach is that it only interposes on value-initialization and not all initializations, ensuring correct default-initialization. This method is particularly beneficial in scenarios where you need a buffer that is large enough to hold data but do not require the buffer to be initialized with any specific value. By employing this custom allocator, unnecessary copy-construction of new values is avoided, leading to more efficient memory usage and potentially better performance in I/O operations.
Here’s a brief code snippet illustrating the concept:
// Allocator adaptor that interposes construct() calls to
// convert value initialization into default initialization.
template <typename T, typename A=std::allocator<T>>
class default_init_allocator : public A {
typedef std::allocator_traits<A> a_t;
public:
template <typename U> struct rebind {
using other =
default_init_allocator<
U, typename a_t::template rebind_alloc<U>
>;
};
using A::A;
template <typename U>
void construct(U* ptr)
noexcept(std::is_nothrow_default_constructible<U>::value) {
::new(static_cast<void*>(ptr)) U;
}
template <typename U, typename...Args>
void construct(U* ptr, Args&&... args) {
a_t::construct(static_cast<A&>(*this),
ptr, std::forward<Args>(args)...);
}
};This custom allocator can be used with std::vector or any other standard container to avoid unnecessary initialization, making it a more accurate and efficient solution for certain use cases in I/O operations:
class io_handler
{
// ... existing code ...
std::vector<char, default_init_allocator<char>> output_buffer_;
public:
// ... existing code ...
// Read data from the input buffer and return true if successful, false otherwise
[[nodiscard]] bool read_data()
{
// ... existing code ...
// Resizing the vector won't cause zero-initialization
output_buffer_.resize(out_bytes);
return result;
}
};In conclusion, it is worth noting that using std::vector as an output buffer for I/O operations can hardly be called an optimal choice, despite its apparent simplicity.
Here’s an example of an io_handler implementation that is both performance-efficient and does not require the tricks that we used in the previous workarounds:
class io_handler
{
static constexpr size_t input_buffer_size = 10 * 1024; // 10KB
static constexpr size_t output_buffer_size = 10 * 1024 * 1024; // 10MB
HANDLE file_handle_{ nullptr };
std::unique_ptr<char[]> input_buffer_;
DWORD input_bytes_size_{ 0 }; // Number of valid bytes in the input buffer
std::unique_ptr<char[]> output_buffer_;
DWORD output_bytes_size_{ 0 }; // Number of valid bytes in the output buffer
public:
io_handler()
{
input_buffer_ = std::make_unique<char[]>(input_buffer_size); // Reserve space in the input buffer
output_buffer_ = std::make_unique<char[]>(output_buffer_size); // Reserve space in the output buffer
// TODO: Create the file handle using ::CreateFile() function
}
~io_handler()
{
CloseHandle(file_handle_); // Close the file handle
}
io_handler(const io_handler& other) = delete;
io_handler(io_handler&& other) = delete;
io_handler& operator=(const io_handler& other) = delete;
io_handler& operator=(io_handler&& other) = delete;
// Read data from the input buffer and return true if successful, false otherwise
[[nodiscard]] bool read_data()
{
const BOOL result = ::DeviceIoControl(
file_handle_, // File handle to use
IOCTL_READ_DATA_CHUNK, // IOCTL code
input_buffer_.get(), // Input buffer data
input_bytes_size_, // Input buffer size
output_buffer_.get(), // Output buffer data
output_buffer_size, // Output buffer size
&output_bytes_size_, // Number of bytes received
nullptr // No overlapped structure
);
// Return true if DeviceIoControl returned non-zero value
return result;
}
};However, in situations where it is necessary to maintain and optimize existing code using std::vector<char>, a specialized template class, such as no_init_primitive_type or providing a custom Allocator::construct, can help developers minimize performance bottlenecks and optimize code for more efficient memory management. The examples provided in this article can serve as reference material for those who want to improve their understanding of memory allocation and initialization in the context of I/O operations using std::vector.