Skip to content

Commit 883d094

Browse files
committed
Fix DALIGenericPeekableIterator missing pmap_compatible parameter.
- The pmap_compatible argument was added to DALIGenericIterator in ff8655b but was not propagated to DALIGenericPeekableIterator, causing test_iterators_init_method_api_compatibility to fail. Signed-off-by: Janusz Lisiecki <jlisiecki@nvidia.com>
1 parent e1fdf02 commit 883d094

1 file changed

Lines changed: 10 additions & 0 deletions

File tree

  • dali/python/nvidia/dali/plugin/jax

dali/python/nvidia/dali/plugin/jax/clu.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,14 @@ class DALIGenericPeekableIterator(DALIGenericIterator):
101101
`jax.sharding.Sharding` compatible object that, if present, will be used to
102102
build an output jax.Array for each category. If ``None``, the iterator returns
103103
values compatible with pmapped JAX functions, if multiple pipelines are provided.
104+
pmap_compatible : bool, optional, default = None
105+
Controls whether the iterator produces outputs with a leading device axis
106+
compatible with ``jax.pmap``. When ``None`` (default), it is inferred
107+
automatically: ``True`` when ``devices`` is provided, ``False`` otherwise.
108+
Set to ``True`` explicitly to force pmap-compatible output (shape
109+
``[num_devices, batch_per_device, ...]``) without using the ``devices``
110+
argument. Set to ``False`` to suppress the device axis even when ``devices``
111+
is provided.
104112
105113
Example
106114
-------
@@ -133,6 +141,7 @@ def __init__(
133141
last_batch_policy: LastBatchPolicy = LastBatchPolicy.FILL,
134142
prepare_first_batch: bool = True,
135143
sharding: Optional[Sharding] = None,
144+
pmap_compatible: Optional[bool] = None,
136145
):
137146
super().__init__(
138147
pipelines,
@@ -144,6 +153,7 @@ def __init__(
144153
last_batch_policy,
145154
prepare_first_batch,
146155
sharding,
156+
pmap_compatible,
147157
)
148158
self._mutex = threading.Lock()
149159
self._pool = None

0 commit comments

Comments
 (0)