wgpu_core/command/
bundle.rs

1/*! Render Bundles
2
3A render bundle is a prerecorded sequence of commands that can be replayed on a
4command encoder with a single call. A single bundle can replayed any number of
5times, on different encoders. Constructing a render bundle lets `wgpu` validate
6and analyze its commands up front, so that replaying a bundle can be more
7efficient than simply re-recording its commands each time.
8
9Not all commands are available in bundles; for example, a render bundle may not
10contain a [`RenderCommand::SetViewport`] command.
11
12Most of `wgpu`'s backend graphics APIs have something like bundles. For example,
13Vulkan calls them "secondary command buffers", and Metal calls them "indirect
14command buffers". Although we plan to take advantage of these platform features
15at some point in the future, for now `wgpu`'s implementation of render bundles
16does not use them: at the hal level, `wgpu` render bundles just replay the
17commands.
18
19## Render Bundle Isolation
20
21One important property of render bundles is that the draw calls in a render
22bundle depend solely on the pipeline and state established within the render
23bundle itself. A draw call in a bundle will never use a vertex buffer, say, that
24was set in the `RenderPass` before executing the bundle. We call this property
25'isolation', in that a render bundle is somewhat isolated from the passes that
26use it.
27
28Render passes are also isolated from the effects of bundles. After executing a
29render bundle, a render pass's pipeline, bind groups, and vertex and index
30buffers are are unset, so the bundle cannot affect later draw calls in the pass.
31
32A render pass is not fully isolated from a bundle's effects on push constant
33values. Draw calls following a bundle's execution will see whatever values the
34bundle writes to push constant storage. Setting a pipeline initializes any push
35constant storage it could access to zero, and this initialization may also be
36visible after bundle execution.
37
38## Render Bundle Lifecycle
39
40To create a render bundle:
41
421) Create a [`RenderBundleEncoder`] by calling
43   [`Global::device_create_render_bundle_encoder`][Gdcrbe].
44
452) Record commands in the `RenderBundleEncoder` using functions from the
46   [`bundle_ffi`] module.
47
483) Call [`Global::render_bundle_encoder_finish`][Grbef], which analyzes and cleans up
49   the command stream and returns a `RenderBundleId`.
50
514) Then, any number of times, call [`render_pass_execute_bundles`][wrpeb] to
52   execute the bundle as part of some render pass.
53
54## Implementation
55
56The most complex part of render bundles is the "finish" step, mostly implemented
57in [`RenderBundleEncoder::finish`]. This consumes the commands stored in the
58encoder's [`BasePass`], while validating everything, tracking the state,
59dropping redundant or unnecessary commands, and presenting the results as a new
60[`RenderBundle`]. It doesn't actually execute any commands.
61
62This step also enforces the 'isolation' property mentioned above: every draw
63call is checked to ensure that the resources it uses on were established since
64the last time the pipeline was set. This means the bundle can be executed
65verbatim without any state tracking.
66
67### Execution
68
69When the bundle is used in an actual render pass, `RenderBundle::execute` is
70called. It goes through the commands and issues them into the native command
71buffer. Thanks to isolation, it doesn't track any bind group invalidations or
72index format changes.
73
74[Gdcrbe]: crate::global::Global::device_create_render_bundle_encoder
75[Grbef]: crate::global::Global::render_bundle_encoder_finish
76[wrpeb]: crate::global::Global::render_pass_execute_bundles
77!*/
78
79#![allow(clippy::reversed_empty_ranges)]
80
81use alloc::{
82    borrow::{Cow, ToOwned as _},
83    string::String,
84    sync::Arc,
85    vec::Vec,
86};
87use core::{
88    num::{NonZeroU32, NonZeroU64},
89    ops::Range,
90};
91
92use arrayvec::ArrayVec;
93use thiserror::Error;
94
95use crate::{
96    binding_model::{BindError, BindGroup, PipelineLayout},
97    command::{
98        BasePass, BindGroupStateChange, ColorAttachmentError, DrawError, MapPassErr,
99        PassErrorScope, RenderCommandError, StateChange,
100    },
101    device::{
102        AttachmentData, Device, DeviceError, MissingDownlevelFlags, RenderPassContext,
103        SHADER_STAGE_COUNT,
104    },
105    hub::Hub,
106    id,
107    init_tracker::{BufferInitTrackerAction, MemoryInitKind, TextureInitTrackerAction},
108    pipeline::{PipelineFlags, RenderPipeline, VertexStep},
109    resource::{
110        Buffer, DestroyedResourceError, Fallible, InvalidResourceError, Labeled, ParentDevice,
111        TrackingData,
112    },
113    resource_log,
114    snatch::SnatchGuard,
115    track::RenderBundleScope,
116    Label, LabelHelpers,
117};
118
119use super::{
120    render_command::{ArcRenderCommand, RenderCommand},
121    DrawKind,
122};
123
124/// Describes a [`RenderBundleEncoder`].
125#[derive(Clone, Debug, Default, PartialEq, Eq, Hash)]
126#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
127pub struct RenderBundleEncoderDescriptor<'a> {
128    /// Debug label of the render bundle encoder.
129    ///
130    /// This will show up in graphics debuggers for easy identification.
131    pub label: Label<'a>,
132    /// The formats of the color attachments that this render bundle is capable
133    /// to rendering to.
134    ///
135    /// This must match the formats of the color attachments in the
136    /// renderpass this render bundle is executed in.
137    pub color_formats: Cow<'a, [Option<wgt::TextureFormat>]>,
138    /// Information about the depth attachment that this render bundle is
139    /// capable to rendering to.
140    ///
141    /// The format must match the format of the depth attachments in the
142    /// renderpass this render bundle is executed in.
143    pub depth_stencil: Option<wgt::RenderBundleDepthStencil>,
144    /// Sample count this render bundle is capable of rendering to.
145    ///
146    /// This must match the pipelines and the renderpasses it is used in.
147    pub sample_count: u32,
148    /// If this render bundle will rendering to multiple array layers in the
149    /// attachments at the same time.
150    pub multiview: Option<NonZeroU32>,
151}
152
153#[derive(Debug)]
154#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
155pub struct RenderBundleEncoder {
156    base: BasePass<RenderCommand>,
157    parent_id: id::DeviceId,
158    pub(crate) context: RenderPassContext,
159    pub(crate) is_depth_read_only: bool,
160    pub(crate) is_stencil_read_only: bool,
161
162    // Resource binding dedupe state.
163    #[cfg_attr(feature = "serde", serde(skip))]
164    current_bind_groups: BindGroupStateChange,
165    #[cfg_attr(feature = "serde", serde(skip))]
166    current_pipeline: StateChange<id::RenderPipelineId>,
167}
168
169impl RenderBundleEncoder {
170    pub fn new(
171        desc: &RenderBundleEncoderDescriptor,
172        parent_id: id::DeviceId,
173        base: Option<BasePass<RenderCommand>>,
174    ) -> Result<Self, CreateRenderBundleError> {
175        let (is_depth_read_only, is_stencil_read_only) = match desc.depth_stencil {
176            Some(ds) => {
177                let aspects = hal::FormatAspects::from(ds.format);
178                (
179                    !aspects.contains(hal::FormatAspects::DEPTH) || ds.depth_read_only,
180                    !aspects.contains(hal::FormatAspects::STENCIL) || ds.stencil_read_only,
181                )
182            }
183            // There's no depth/stencil attachment, so these values just don't
184            // matter.  Choose the most accommodating value, to simplify
185            // validation.
186            None => (true, true),
187        };
188
189        // TODO: should be device.limits.max_color_attachments
190        let max_color_attachments = hal::MAX_COLOR_ATTACHMENTS;
191
192        //TODO: validate that attachment formats are renderable,
193        // have expected aspects, support multisampling.
194        Ok(Self {
195            base: base.unwrap_or_else(|| BasePass::new(&desc.label)),
196            parent_id,
197            context: RenderPassContext {
198                attachments: AttachmentData {
199                    colors: if desc.color_formats.len() > max_color_attachments {
200                        return Err(CreateRenderBundleError::ColorAttachment(
201                            ColorAttachmentError::TooMany {
202                                given: desc.color_formats.len(),
203                                limit: max_color_attachments,
204                            },
205                        ));
206                    } else {
207                        desc.color_formats.iter().cloned().collect()
208                    },
209                    resolves: ArrayVec::new(),
210                    depth_stencil: desc.depth_stencil.map(|ds| ds.format),
211                },
212                sample_count: {
213                    let sc = desc.sample_count;
214                    if sc == 0 || sc > 32 || !sc.is_power_of_two() {
215                        return Err(CreateRenderBundleError::InvalidSampleCount(sc));
216                    }
217                    sc
218                },
219                multiview: desc.multiview,
220            },
221
222            is_depth_read_only,
223            is_stencil_read_only,
224            current_bind_groups: BindGroupStateChange::new(),
225            current_pipeline: StateChange::new(),
226        })
227    }
228
229    pub fn dummy(parent_id: id::DeviceId) -> Self {
230        Self {
231            base: BasePass::new(&None),
232            parent_id,
233            context: RenderPassContext {
234                attachments: AttachmentData {
235                    colors: ArrayVec::new(),
236                    resolves: ArrayVec::new(),
237                    depth_stencil: None,
238                },
239                sample_count: 0,
240                multiview: None,
241            },
242            is_depth_read_only: false,
243            is_stencil_read_only: false,
244
245            current_bind_groups: BindGroupStateChange::new(),
246            current_pipeline: StateChange::new(),
247        }
248    }
249
250    #[cfg(feature = "trace")]
251    pub(crate) fn to_base_pass(&self) -> BasePass<RenderCommand> {
252        self.base.clone()
253    }
254
255    pub fn parent(&self) -> id::DeviceId {
256        self.parent_id
257    }
258
259    /// Convert this encoder's commands into a [`RenderBundle`].
260    ///
261    /// We want executing a [`RenderBundle`] to be quick, so we take
262    /// this opportunity to clean up the [`RenderBundleEncoder`]'s
263    /// command stream and gather metadata about it that will help
264    /// keep [`ExecuteBundle`] simple and fast. We remove redundant
265    /// commands (along with their side data), note resource usage,
266    /// and accumulate buffer and texture initialization actions.
267    ///
268    /// [`ExecuteBundle`]: RenderCommand::ExecuteBundle
269    pub(crate) fn finish(
270        self,
271        desc: &RenderBundleDescriptor,
272        device: &Arc<Device>,
273        hub: &Hub,
274    ) -> Result<Arc<RenderBundle>, RenderBundleError> {
275        let scope = PassErrorScope::Bundle;
276
277        device.check_is_valid().map_pass_err(scope)?;
278
279        let bind_group_guard = hub.bind_groups.read();
280        let pipeline_guard = hub.render_pipelines.read();
281        let buffer_guard = hub.buffers.read();
282
283        let mut state = State {
284            trackers: RenderBundleScope::new(),
285            pipeline: None,
286            bind: (0..hal::MAX_BIND_GROUPS).map(|_| None).collect(),
287            vertex: Default::default(),
288            index: None,
289            flat_dynamic_offsets: Vec::new(),
290            device: device.clone(),
291            commands: Vec::new(),
292            buffer_memory_init_actions: Vec::new(),
293            texture_memory_init_actions: Vec::new(),
294            next_dynamic_offset: 0,
295        };
296
297        let indices = &state.device.tracker_indices;
298        state.trackers.buffers.set_size(indices.buffers.size());
299        state.trackers.textures.set_size(indices.textures.size());
300
301        let base = &self.base;
302
303        for &command in &base.commands {
304            match command {
305                RenderCommand::SetBindGroup {
306                    index,
307                    num_dynamic_offsets,
308                    bind_group_id,
309                } => {
310                    let scope = PassErrorScope::SetBindGroup;
311                    set_bind_group(
312                        &mut state,
313                        &bind_group_guard,
314                        &base.dynamic_offsets,
315                        index,
316                        num_dynamic_offsets,
317                        bind_group_id,
318                    )
319                    .map_pass_err(scope)?;
320                }
321                RenderCommand::SetPipeline(pipeline_id) => {
322                    let scope = PassErrorScope::SetPipelineRender;
323                    set_pipeline(
324                        &mut state,
325                        &pipeline_guard,
326                        &self.context,
327                        self.is_depth_read_only,
328                        self.is_stencil_read_only,
329                        pipeline_id,
330                    )
331                    .map_pass_err(scope)?;
332                }
333                RenderCommand::SetIndexBuffer {
334                    buffer_id,
335                    index_format,
336                    offset,
337                    size,
338                } => {
339                    let scope = PassErrorScope::SetIndexBuffer;
340                    set_index_buffer(
341                        &mut state,
342                        &buffer_guard,
343                        buffer_id,
344                        index_format,
345                        offset,
346                        size,
347                    )
348                    .map_pass_err(scope)?;
349                }
350                RenderCommand::SetVertexBuffer {
351                    slot,
352                    buffer_id,
353                    offset,
354                    size,
355                } => {
356                    let scope = PassErrorScope::SetVertexBuffer;
357                    set_vertex_buffer(&mut state, &buffer_guard, slot, buffer_id, offset, size)
358                        .map_pass_err(scope)?;
359                }
360                RenderCommand::SetPushConstant {
361                    stages,
362                    offset,
363                    size_bytes,
364                    values_offset,
365                } => {
366                    let scope = PassErrorScope::SetPushConstant;
367                    set_push_constant(&mut state, stages, offset, size_bytes, values_offset)
368                        .map_pass_err(scope)?;
369                }
370                RenderCommand::Draw {
371                    vertex_count,
372                    instance_count,
373                    first_vertex,
374                    first_instance,
375                } => {
376                    let scope = PassErrorScope::Draw {
377                        kind: DrawKind::Draw,
378                        indexed: false,
379                    };
380                    draw(
381                        &mut state,
382                        &base.dynamic_offsets,
383                        vertex_count,
384                        instance_count,
385                        first_vertex,
386                        first_instance,
387                    )
388                    .map_pass_err(scope)?;
389                }
390                RenderCommand::DrawIndexed {
391                    index_count,
392                    instance_count,
393                    first_index,
394                    base_vertex,
395                    first_instance,
396                } => {
397                    let scope = PassErrorScope::Draw {
398                        kind: DrawKind::Draw,
399                        indexed: true,
400                    };
401                    draw_indexed(
402                        &mut state,
403                        &base.dynamic_offsets,
404                        index_count,
405                        instance_count,
406                        first_index,
407                        base_vertex,
408                        first_instance,
409                    )
410                    .map_pass_err(scope)?;
411                }
412                RenderCommand::DrawIndirect {
413                    buffer_id,
414                    offset,
415                    count: 1,
416                    indexed,
417                } => {
418                    let scope = PassErrorScope::Draw {
419                        kind: DrawKind::DrawIndirect,
420                        indexed,
421                    };
422                    multi_draw_indirect(
423                        &mut state,
424                        &base.dynamic_offsets,
425                        &buffer_guard,
426                        buffer_id,
427                        offset,
428                        indexed,
429                    )
430                    .map_pass_err(scope)?;
431                }
432                RenderCommand::DrawIndirect { .. }
433                | RenderCommand::MultiDrawIndirectCount { .. } => unimplemented!(),
434                RenderCommand::PushDebugGroup { color: _, len: _ } => unimplemented!(),
435                RenderCommand::InsertDebugMarker { color: _, len: _ } => unimplemented!(),
436                RenderCommand::PopDebugGroup => unimplemented!(),
437                // Must check the TIMESTAMP_QUERY_INSIDE_PASSES feature
438                RenderCommand::WriteTimestamp { .. }
439                | RenderCommand::BeginOcclusionQuery { .. }
440                | RenderCommand::EndOcclusionQuery
441                | RenderCommand::BeginPipelineStatisticsQuery { .. }
442                | RenderCommand::EndPipelineStatisticsQuery => unimplemented!(),
443                RenderCommand::ExecuteBundle(_)
444                | RenderCommand::SetBlendConstant(_)
445                | RenderCommand::SetStencilReference(_)
446                | RenderCommand::SetViewport { .. }
447                | RenderCommand::SetScissor(_) => unreachable!("not supported by a render bundle"),
448            }
449        }
450
451        let State {
452            trackers,
453            flat_dynamic_offsets,
454            device,
455            commands,
456            buffer_memory_init_actions,
457            texture_memory_init_actions,
458            ..
459        } = state;
460
461        let tracker_indices = device.tracker_indices.bundles.clone();
462        let discard_hal_labels = device
463            .instance_flags
464            .contains(wgt::InstanceFlags::DISCARD_HAL_LABELS);
465
466        let render_bundle = RenderBundle {
467            base: BasePass {
468                label: desc.label.as_deref().map(str::to_owned),
469                commands,
470                dynamic_offsets: flat_dynamic_offsets,
471                string_data: self.base.string_data,
472                push_constant_data: self.base.push_constant_data,
473            },
474            is_depth_read_only: self.is_depth_read_only,
475            is_stencil_read_only: self.is_stencil_read_only,
476            device: device.clone(),
477            used: trackers,
478            buffer_memory_init_actions,
479            texture_memory_init_actions,
480            context: self.context,
481            label: desc.label.to_string(),
482            tracking_data: TrackingData::new(tracker_indices),
483            discard_hal_labels,
484        };
485
486        let render_bundle = Arc::new(render_bundle);
487
488        Ok(render_bundle)
489    }
490
491    pub fn set_index_buffer(
492        &mut self,
493        buffer_id: id::BufferId,
494        index_format: wgt::IndexFormat,
495        offset: wgt::BufferAddress,
496        size: Option<wgt::BufferSize>,
497    ) {
498        self.base.commands.push(RenderCommand::SetIndexBuffer {
499            buffer_id,
500            index_format,
501            offset,
502            size,
503        });
504    }
505}
506
507fn set_bind_group(
508    state: &mut State,
509    bind_group_guard: &crate::storage::Storage<Fallible<BindGroup>>,
510    dynamic_offsets: &[u32],
511    index: u32,
512    num_dynamic_offsets: usize,
513    bind_group_id: Option<id::Id<id::markers::BindGroup>>,
514) -> Result<(), RenderBundleErrorInner> {
515    if bind_group_id.is_none() {
516        // TODO: do appropriate cleanup for null bind_group.
517        return Ok(());
518    }
519
520    let bind_group_id = bind_group_id.unwrap();
521
522    let bind_group = bind_group_guard.get(bind_group_id).get()?;
523
524    bind_group.same_device(&state.device)?;
525
526    let max_bind_groups = state.device.limits.max_bind_groups;
527    if index >= max_bind_groups {
528        return Err(RenderCommandError::BindGroupIndexOutOfRange {
529            index,
530            max: max_bind_groups,
531        }
532        .into());
533    }
534
535    // Identify the next `num_dynamic_offsets` entries from `dynamic_offsets`.
536    let offsets_range = state.next_dynamic_offset..state.next_dynamic_offset + num_dynamic_offsets;
537    state.next_dynamic_offset = offsets_range.end;
538    let offsets = &dynamic_offsets[offsets_range.clone()];
539
540    bind_group.validate_dynamic_bindings(index, offsets)?;
541
542    state
543        .buffer_memory_init_actions
544        .extend_from_slice(&bind_group.used_buffer_ranges);
545    state
546        .texture_memory_init_actions
547        .extend_from_slice(&bind_group.used_texture_ranges);
548
549    state.set_bind_group(index, &bind_group, offsets_range);
550    unsafe { state.trackers.merge_bind_group(&bind_group.used)? };
551    state.trackers.bind_groups.insert_single(bind_group);
552    // Note: stateless trackers are not merged: the lifetime reference
553    // is held to the bind group itself.
554    Ok(())
555}
556
557fn set_pipeline(
558    state: &mut State,
559    pipeline_guard: &crate::storage::Storage<Fallible<RenderPipeline>>,
560    context: &RenderPassContext,
561    is_depth_read_only: bool,
562    is_stencil_read_only: bool,
563    pipeline_id: id::Id<id::markers::RenderPipeline>,
564) -> Result<(), RenderBundleErrorInner> {
565    let pipeline = pipeline_guard.get(pipeline_id).get()?;
566
567    pipeline.same_device(&state.device)?;
568
569    context
570        .check_compatible(&pipeline.pass_context, pipeline.as_ref())
571        .map_err(RenderCommandError::IncompatiblePipelineTargets)?;
572
573    if pipeline.flags.contains(PipelineFlags::WRITES_DEPTH) && is_depth_read_only {
574        return Err(RenderCommandError::IncompatibleDepthAccess(pipeline.error_ident()).into());
575    }
576    if pipeline.flags.contains(PipelineFlags::WRITES_STENCIL) && is_stencil_read_only {
577        return Err(RenderCommandError::IncompatibleStencilAccess(pipeline.error_ident()).into());
578    }
579
580    let pipeline_state = PipelineState::new(&pipeline);
581
582    state
583        .commands
584        .push(ArcRenderCommand::SetPipeline(pipeline.clone()));
585
586    // If this pipeline uses push constants, zero out their values.
587    if let Some(iter) = pipeline_state.zero_push_constants() {
588        state.commands.extend(iter)
589    }
590
591    state.invalidate_bind_groups(&pipeline_state, &pipeline.layout);
592    state.pipeline = Some(pipeline_state);
593
594    state.trackers.render_pipelines.insert_single(pipeline);
595    Ok(())
596}
597
598fn set_index_buffer(
599    state: &mut State,
600    buffer_guard: &crate::storage::Storage<Fallible<Buffer>>,
601    buffer_id: id::Id<id::markers::Buffer>,
602    index_format: wgt::IndexFormat,
603    offset: u64,
604    size: Option<NonZeroU64>,
605) -> Result<(), RenderBundleErrorInner> {
606    let buffer = buffer_guard.get(buffer_id).get()?;
607
608    state
609        .trackers
610        .buffers
611        .merge_single(&buffer, wgt::BufferUses::INDEX)?;
612
613    buffer.same_device(&state.device)?;
614    buffer.check_usage(wgt::BufferUsages::INDEX)?;
615
616    let end = match size {
617        Some(s) => offset + s.get(),
618        None => buffer.size,
619    };
620    state
621        .buffer_memory_init_actions
622        .extend(buffer.initialization_status.read().create_action(
623            &buffer,
624            offset..end,
625            MemoryInitKind::NeedsInitializedMemory,
626        ));
627    state.set_index_buffer(buffer, index_format, offset..end);
628    Ok(())
629}
630
631fn set_vertex_buffer(
632    state: &mut State,
633    buffer_guard: &crate::storage::Storage<Fallible<Buffer>>,
634    slot: u32,
635    buffer_id: id::Id<id::markers::Buffer>,
636    offset: u64,
637    size: Option<NonZeroU64>,
638) -> Result<(), RenderBundleErrorInner> {
639    let max_vertex_buffers = state.device.limits.max_vertex_buffers;
640    if slot >= max_vertex_buffers {
641        return Err(RenderCommandError::VertexBufferIndexOutOfRange {
642            index: slot,
643            max: max_vertex_buffers,
644        }
645        .into());
646    }
647
648    let buffer = buffer_guard.get(buffer_id).get()?;
649
650    state
651        .trackers
652        .buffers
653        .merge_single(&buffer, wgt::BufferUses::VERTEX)?;
654
655    buffer.same_device(&state.device)?;
656    buffer.check_usage(wgt::BufferUsages::VERTEX)?;
657
658    let end = match size {
659        Some(s) => offset + s.get(),
660        None => buffer.size,
661    };
662    state
663        .buffer_memory_init_actions
664        .extend(buffer.initialization_status.read().create_action(
665            &buffer,
666            offset..end,
667            MemoryInitKind::NeedsInitializedMemory,
668        ));
669    state.vertex[slot as usize] = Some(VertexState::new(buffer, offset..end));
670    Ok(())
671}
672
673fn set_push_constant(
674    state: &mut State,
675    stages: wgt::ShaderStages,
676    offset: u32,
677    size_bytes: u32,
678    values_offset: Option<u32>,
679) -> Result<(), RenderBundleErrorInner> {
680    let end_offset = offset + size_bytes;
681
682    let pipeline_state = state.pipeline()?;
683
684    pipeline_state
685        .pipeline
686        .layout
687        .validate_push_constant_ranges(stages, offset, end_offset)?;
688
689    state.commands.push(ArcRenderCommand::SetPushConstant {
690        stages,
691        offset,
692        size_bytes,
693        values_offset,
694    });
695    Ok(())
696}
697
698fn draw(
699    state: &mut State,
700    dynamic_offsets: &[u32],
701    vertex_count: u32,
702    instance_count: u32,
703    first_vertex: u32,
704    first_instance: u32,
705) -> Result<(), RenderBundleErrorInner> {
706    let pipeline = state.pipeline()?;
707    let used_bind_groups = pipeline.used_bind_groups;
708
709    let vertex_limits = super::VertexLimits::new(state.vertex_buffer_sizes(), &pipeline.steps);
710    vertex_limits.validate_vertex_limit(first_vertex, vertex_count)?;
711    vertex_limits.validate_instance_limit(first_instance, instance_count)?;
712
713    if instance_count > 0 && vertex_count > 0 {
714        state.flush_vertices();
715        state.flush_binds(used_bind_groups, dynamic_offsets);
716        state.commands.push(ArcRenderCommand::Draw {
717            vertex_count,
718            instance_count,
719            first_vertex,
720            first_instance,
721        });
722    }
723    Ok(())
724}
725
726fn draw_indexed(
727    state: &mut State,
728    dynamic_offsets: &[u32],
729    index_count: u32,
730    instance_count: u32,
731    first_index: u32,
732    base_vertex: i32,
733    first_instance: u32,
734) -> Result<(), RenderBundleErrorInner> {
735    let pipeline = state.pipeline()?;
736    let used_bind_groups = pipeline.used_bind_groups;
737    let index = match state.index {
738        Some(ref index) => index,
739        None => return Err(DrawError::MissingIndexBuffer.into()),
740    };
741
742    let vertex_limits = super::VertexLimits::new(state.vertex_buffer_sizes(), &pipeline.steps);
743
744    let last_index = first_index as u64 + index_count as u64;
745    let index_limit = index.limit();
746    if last_index > index_limit {
747        return Err(DrawError::IndexBeyondLimit {
748            last_index,
749            index_limit,
750        }
751        .into());
752    }
753    vertex_limits.validate_instance_limit(first_instance, instance_count)?;
754
755    if instance_count > 0 && index_count > 0 {
756        state.flush_index();
757        state.flush_vertices();
758        state.flush_binds(used_bind_groups, dynamic_offsets);
759        state.commands.push(ArcRenderCommand::DrawIndexed {
760            index_count,
761            instance_count,
762            first_index,
763            base_vertex,
764            first_instance,
765        });
766    }
767    Ok(())
768}
769
770fn multi_draw_indirect(
771    state: &mut State,
772    dynamic_offsets: &[u32],
773    buffer_guard: &crate::storage::Storage<Fallible<Buffer>>,
774    buffer_id: id::Id<id::markers::Buffer>,
775    offset: u64,
776    indexed: bool,
777) -> Result<(), RenderBundleErrorInner> {
778    state
779        .device
780        .require_downlevel_flags(wgt::DownlevelFlags::INDIRECT_EXECUTION)?;
781
782    let pipeline = state.pipeline()?;
783    let used_bind_groups = pipeline.used_bind_groups;
784
785    let buffer = buffer_guard.get(buffer_id).get()?;
786
787    buffer.same_device(&state.device)?;
788    buffer.check_usage(wgt::BufferUsages::INDIRECT)?;
789
790    let vertex_limits = super::VertexLimits::new(state.vertex_buffer_sizes(), &pipeline.steps);
791
792    let stride = super::get_stride_of_indirect_args(indexed);
793    state
794        .buffer_memory_init_actions
795        .extend(buffer.initialization_status.read().create_action(
796            &buffer,
797            offset..(offset + stride),
798            MemoryInitKind::NeedsInitializedMemory,
799        ));
800
801    let vertex_or_index_limit = if indexed {
802        let index = match state.index {
803            Some(ref mut index) => index,
804            None => return Err(DrawError::MissingIndexBuffer.into()),
805        };
806        state.commands.extend(index.flush());
807        index.limit()
808    } else {
809        vertex_limits.vertex_limit
810    };
811    let instance_limit = vertex_limits.instance_limit;
812
813    let buffer_uses = if state.device.indirect_validation.is_some() {
814        wgt::BufferUses::STORAGE_READ_ONLY
815    } else {
816        wgt::BufferUses::INDIRECT
817    };
818
819    state.trackers.buffers.merge_single(&buffer, buffer_uses)?;
820
821    state.flush_vertices();
822    state.flush_binds(used_bind_groups, dynamic_offsets);
823    state.commands.push(ArcRenderCommand::DrawIndirect {
824        buffer,
825        offset,
826        count: 1,
827        indexed,
828
829        vertex_or_index_limit,
830        instance_limit,
831    });
832    Ok(())
833}
834
835/// Error type returned from `RenderBundleEncoder::new` if the sample count is invalid.
836#[derive(Clone, Debug, Error)]
837#[non_exhaustive]
838pub enum CreateRenderBundleError {
839    #[error(transparent)]
840    ColorAttachment(#[from] ColorAttachmentError),
841    #[error("Invalid number of samples {0}")]
842    InvalidSampleCount(u32),
843}
844
845/// Error type returned from `RenderBundleEncoder::new` if the sample count is invalid.
846#[derive(Clone, Debug, Error)]
847#[non_exhaustive]
848pub enum ExecutionError {
849    #[error(transparent)]
850    Device(#[from] DeviceError),
851    #[error(transparent)]
852    DestroyedResource(#[from] DestroyedResourceError),
853    #[error("Using {0} in a render bundle is not implemented")]
854    Unimplemented(&'static str),
855}
856
857pub type RenderBundleDescriptor<'a> = wgt::RenderBundleDescriptor<Label<'a>>;
858
859//Note: here, `RenderBundle` is just wrapping a raw stream of render commands.
860// The plan is to back it by an actual Vulkan secondary buffer, D3D12 Bundle,
861// or Metal indirect command buffer.
862#[derive(Debug)]
863pub struct RenderBundle {
864    // Normalized command stream. It can be executed verbatim,
865    // without re-binding anything on the pipeline change.
866    base: BasePass<ArcRenderCommand>,
867    pub(super) is_depth_read_only: bool,
868    pub(super) is_stencil_read_only: bool,
869    pub(crate) device: Arc<Device>,
870    pub(crate) used: RenderBundleScope,
871    pub(super) buffer_memory_init_actions: Vec<BufferInitTrackerAction>,
872    pub(super) texture_memory_init_actions: Vec<TextureInitTrackerAction>,
873    pub(super) context: RenderPassContext,
874    /// The `label` from the descriptor used to create the resource.
875    label: String,
876    pub(crate) tracking_data: TrackingData,
877    discard_hal_labels: bool,
878}
879
880impl Drop for RenderBundle {
881    fn drop(&mut self) {
882        resource_log!("Drop {}", self.error_ident());
883    }
884}
885
886#[cfg(send_sync)]
887unsafe impl Send for RenderBundle {}
888#[cfg(send_sync)]
889unsafe impl Sync for RenderBundle {}
890
891impl RenderBundle {
892    /// Actually encode the contents into a native command buffer.
893    ///
894    /// This is partially duplicating the logic of `render_pass_end`.
895    /// However the point of this function is to be lighter, since we already had
896    /// a chance to go through the commands in `render_bundle_encoder_finish`.
897    ///
898    /// Note that the function isn't expected to fail, generally.
899    /// All the validation has already been done by this point.
900    /// The only failure condition is if some of the used buffers are destroyed.
901    pub(super) unsafe fn execute(
902        &self,
903        raw: &mut dyn hal::DynCommandEncoder,
904        indirect_draw_validation_resources: &mut crate::indirect_validation::DrawResources,
905        indirect_draw_validation_batcher: &mut crate::indirect_validation::DrawBatcher,
906        snatch_guard: &SnatchGuard,
907    ) -> Result<(), ExecutionError> {
908        let mut offsets = self.base.dynamic_offsets.as_slice();
909        let mut pipeline_layout = None::<Arc<PipelineLayout>>;
910        if !self.discard_hal_labels {
911            if let Some(ref label) = self.base.label {
912                unsafe { raw.begin_debug_marker(label) };
913            }
914        }
915
916        use ArcRenderCommand as Cmd;
917        for command in self.base.commands.iter() {
918            match command {
919                Cmd::SetBindGroup {
920                    index,
921                    num_dynamic_offsets,
922                    bind_group,
923                } => {
924                    let mut bg = None;
925                    if bind_group.is_some() {
926                        let bind_group = bind_group.as_ref().unwrap();
927                        let raw_bg = bind_group.try_raw(snatch_guard)?;
928                        bg = Some(raw_bg);
929                    }
930                    unsafe {
931                        raw.set_bind_group(
932                            pipeline_layout.as_ref().unwrap().raw(),
933                            *index,
934                            bg,
935                            &offsets[..*num_dynamic_offsets],
936                        )
937                    };
938                    offsets = &offsets[*num_dynamic_offsets..];
939                }
940                Cmd::SetPipeline(pipeline) => {
941                    unsafe { raw.set_render_pipeline(pipeline.raw()) };
942
943                    pipeline_layout = Some(pipeline.layout.clone());
944                }
945                Cmd::SetIndexBuffer {
946                    buffer,
947                    index_format,
948                    offset,
949                    size,
950                } => {
951                    let buffer = buffer.try_raw(snatch_guard)?;
952                    let bb = hal::BufferBinding {
953                        buffer,
954                        offset: *offset,
955                        size: *size,
956                    };
957                    unsafe { raw.set_index_buffer(bb, *index_format) };
958                }
959                Cmd::SetVertexBuffer {
960                    slot,
961                    buffer,
962                    offset,
963                    size,
964                } => {
965                    let buffer = buffer.try_raw(snatch_guard)?;
966                    let bb = hal::BufferBinding {
967                        buffer,
968                        offset: *offset,
969                        size: *size,
970                    };
971                    unsafe { raw.set_vertex_buffer(*slot, bb) };
972                }
973                Cmd::SetPushConstant {
974                    stages,
975                    offset,
976                    size_bytes,
977                    values_offset,
978                } => {
979                    let pipeline_layout = pipeline_layout.as_ref().unwrap();
980
981                    if let Some(values_offset) = *values_offset {
982                        let values_end_offset =
983                            (values_offset + size_bytes / wgt::PUSH_CONSTANT_ALIGNMENT) as usize;
984                        let data_slice = &self.base.push_constant_data
985                            [(values_offset as usize)..values_end_offset];
986
987                        unsafe {
988                            raw.set_push_constants(
989                                pipeline_layout.raw(),
990                                *stages,
991                                *offset,
992                                data_slice,
993                            )
994                        }
995                    } else {
996                        super::push_constant_clear(
997                            *offset,
998                            *size_bytes,
999                            |clear_offset, clear_data| {
1000                                unsafe {
1001                                    raw.set_push_constants(
1002                                        pipeline_layout.raw(),
1003                                        *stages,
1004                                        clear_offset,
1005                                        clear_data,
1006                                    )
1007                                };
1008                            },
1009                        );
1010                    }
1011                }
1012                Cmd::Draw {
1013                    vertex_count,
1014                    instance_count,
1015                    first_vertex,
1016                    first_instance,
1017                } => {
1018                    unsafe {
1019                        raw.draw(
1020                            *first_vertex,
1021                            *vertex_count,
1022                            *first_instance,
1023                            *instance_count,
1024                        )
1025                    };
1026                }
1027                Cmd::DrawIndexed {
1028                    index_count,
1029                    instance_count,
1030                    first_index,
1031                    base_vertex,
1032                    first_instance,
1033                } => {
1034                    unsafe {
1035                        raw.draw_indexed(
1036                            *first_index,
1037                            *index_count,
1038                            *base_vertex,
1039                            *first_instance,
1040                            *instance_count,
1041                        )
1042                    };
1043                }
1044                Cmd::DrawIndirect {
1045                    buffer,
1046                    offset,
1047                    count: 1,
1048                    indexed,
1049
1050                    vertex_or_index_limit,
1051                    instance_limit,
1052                } => {
1053                    let (buffer, offset) = if self.device.indirect_validation.is_some() {
1054                        let (dst_resource_index, offset) = indirect_draw_validation_batcher.add(
1055                            indirect_draw_validation_resources,
1056                            &self.device,
1057                            buffer,
1058                            *offset,
1059                            *indexed,
1060                            *vertex_or_index_limit,
1061                            *instance_limit,
1062                        )?;
1063
1064                        let dst_buffer =
1065                            indirect_draw_validation_resources.get_dst_buffer(dst_resource_index);
1066                        (dst_buffer, offset)
1067                    } else {
1068                        (buffer.try_raw(snatch_guard)?, *offset)
1069                    };
1070                    if *indexed {
1071                        unsafe { raw.draw_indexed_indirect(buffer, offset, 1) };
1072                    } else {
1073                        unsafe { raw.draw_indirect(buffer, offset, 1) };
1074                    }
1075                }
1076                Cmd::DrawIndirect { .. } | Cmd::MultiDrawIndirectCount { .. } => {
1077                    return Err(ExecutionError::Unimplemented("multi-draw-indirect"))
1078                }
1079                Cmd::PushDebugGroup { .. } | Cmd::InsertDebugMarker { .. } | Cmd::PopDebugGroup => {
1080                    return Err(ExecutionError::Unimplemented("debug-markers"))
1081                }
1082                Cmd::WriteTimestamp { .. }
1083                | Cmd::BeginOcclusionQuery { .. }
1084                | Cmd::EndOcclusionQuery
1085                | Cmd::BeginPipelineStatisticsQuery { .. }
1086                | Cmd::EndPipelineStatisticsQuery => {
1087                    return Err(ExecutionError::Unimplemented("queries"))
1088                }
1089                Cmd::ExecuteBundle(_)
1090                | Cmd::SetBlendConstant(_)
1091                | Cmd::SetStencilReference(_)
1092                | Cmd::SetViewport { .. }
1093                | Cmd::SetScissor(_) => unreachable!(),
1094            }
1095        }
1096
1097        if !self.discard_hal_labels {
1098            if let Some(_) = self.base.label {
1099                unsafe { raw.end_debug_marker() };
1100            }
1101        }
1102
1103        Ok(())
1104    }
1105}
1106
1107crate::impl_resource_type!(RenderBundle);
1108crate::impl_labeled!(RenderBundle);
1109crate::impl_parent_device!(RenderBundle);
1110crate::impl_storage_item!(RenderBundle);
1111crate::impl_trackable!(RenderBundle);
1112
1113/// A render bundle's current index buffer state.
1114///
1115/// [`RenderBundleEncoder::finish`] records the currently set index buffer here,
1116/// and calls [`State::flush_index`] before any indexed draw command to produce
1117/// a `SetIndexBuffer` command if one is necessary.
1118#[derive(Debug)]
1119struct IndexState {
1120    buffer: Arc<Buffer>,
1121    format: wgt::IndexFormat,
1122    range: Range<wgt::BufferAddress>,
1123    is_dirty: bool,
1124}
1125
1126impl IndexState {
1127    /// Return the number of entries in the current index buffer.
1128    ///
1129    /// Panic if no index buffer has been set.
1130    fn limit(&self) -> u64 {
1131        let bytes_per_index = self.format.byte_size() as u64;
1132
1133        (self.range.end - self.range.start) / bytes_per_index
1134    }
1135
1136    /// Generate a `SetIndexBuffer` command to prepare for an indexed draw
1137    /// command, if needed.
1138    fn flush(&mut self) -> Option<ArcRenderCommand> {
1139        if self.is_dirty {
1140            self.is_dirty = false;
1141            Some(ArcRenderCommand::SetIndexBuffer {
1142                buffer: self.buffer.clone(),
1143                index_format: self.format,
1144                offset: self.range.start,
1145                size: wgt::BufferSize::new(self.range.end - self.range.start),
1146            })
1147        } else {
1148            None
1149        }
1150    }
1151}
1152
1153/// The state of a single vertex buffer slot during render bundle encoding.
1154///
1155/// [`RenderBundleEncoder::finish`] uses this to drop redundant
1156/// `SetVertexBuffer` commands from the final [`RenderBundle`]. It
1157/// records one vertex buffer slot's state changes here, and then
1158/// calls this type's [`flush`] method just before any draw command to
1159/// produce a `SetVertexBuffer` commands if one is necessary.
1160///
1161/// [`flush`]: IndexState::flush
1162#[derive(Debug)]
1163struct VertexState {
1164    buffer: Arc<Buffer>,
1165    range: Range<wgt::BufferAddress>,
1166    is_dirty: bool,
1167}
1168
1169impl VertexState {
1170    fn new(buffer: Arc<Buffer>, range: Range<wgt::BufferAddress>) -> Self {
1171        Self {
1172            buffer,
1173            range,
1174            is_dirty: true,
1175        }
1176    }
1177
1178    /// Generate a `SetVertexBuffer` command for this slot, if necessary.
1179    ///
1180    /// `slot` is the index of the vertex buffer slot that `self` tracks.
1181    fn flush(&mut self, slot: u32) -> Option<ArcRenderCommand> {
1182        if self.is_dirty {
1183            self.is_dirty = false;
1184            Some(ArcRenderCommand::SetVertexBuffer {
1185                slot,
1186                buffer: self.buffer.clone(),
1187                offset: self.range.start,
1188                size: wgt::BufferSize::new(self.range.end - self.range.start),
1189            })
1190        } else {
1191            None
1192        }
1193    }
1194}
1195
1196/// A bind group that has been set at a particular index during render bundle encoding.
1197#[derive(Debug)]
1198struct BindState {
1199    /// The id of the bind group set at this index.
1200    bind_group: Arc<BindGroup>,
1201
1202    /// The range of dynamic offsets for this bind group, in the original
1203    /// command stream's `BassPass::dynamic_offsets` array.
1204    dynamic_offsets: Range<usize>,
1205
1206    /// True if this index's contents have been changed since the last time we
1207    /// generated a `SetBindGroup` command.
1208    is_dirty: bool,
1209}
1210
1211/// The bundle's current pipeline, and some cached information needed for validation.
1212struct PipelineState {
1213    /// The pipeline
1214    pipeline: Arc<RenderPipeline>,
1215
1216    /// How this pipeline's vertex shader traverses each vertex buffer, indexed
1217    /// by vertex buffer slot number.
1218    steps: Vec<VertexStep>,
1219
1220    /// Ranges of push constants this pipeline uses, copied from the pipeline
1221    /// layout.
1222    push_constant_ranges: ArrayVec<wgt::PushConstantRange, { SHADER_STAGE_COUNT }>,
1223
1224    /// The number of bind groups this pipeline uses.
1225    used_bind_groups: usize,
1226}
1227
1228impl PipelineState {
1229    fn new(pipeline: &Arc<RenderPipeline>) -> Self {
1230        Self {
1231            pipeline: pipeline.clone(),
1232            steps: pipeline.vertex_steps.to_vec(),
1233            push_constant_ranges: pipeline
1234                .layout
1235                .push_constant_ranges
1236                .iter()
1237                .cloned()
1238                .collect(),
1239            used_bind_groups: pipeline.layout.bind_group_layouts.len(),
1240        }
1241    }
1242
1243    /// Return a sequence of commands to zero the push constant ranges this
1244    /// pipeline uses. If no initialization is necessary, return `None`.
1245    fn zero_push_constants(&self) -> Option<impl Iterator<Item = ArcRenderCommand>> {
1246        if !self.push_constant_ranges.is_empty() {
1247            let nonoverlapping_ranges =
1248                super::bind::compute_nonoverlapping_ranges(&self.push_constant_ranges);
1249
1250            Some(
1251                nonoverlapping_ranges
1252                    .into_iter()
1253                    .map(|range| ArcRenderCommand::SetPushConstant {
1254                        stages: range.stages,
1255                        offset: range.range.start,
1256                        size_bytes: range.range.end - range.range.start,
1257                        values_offset: None, // write zeros
1258                    }),
1259            )
1260        } else {
1261            None
1262        }
1263    }
1264}
1265
1266/// State for analyzing and cleaning up bundle command streams.
1267///
1268/// To minimize state updates, [`RenderBundleEncoder::finish`]
1269/// actually just applies commands like [`SetBindGroup`] and
1270/// [`SetIndexBuffer`] to the simulated state stored here, and then
1271/// calls the `flush_foo` methods before draw calls to produce the
1272/// update commands we actually need.
1273///
1274/// [`SetBindGroup`]: RenderCommand::SetBindGroup
1275/// [`SetIndexBuffer`]: RenderCommand::SetIndexBuffer
1276struct State {
1277    /// Resources used by this bundle. This will become [`RenderBundle::used`].
1278    trackers: RenderBundleScope,
1279
1280    /// The currently set pipeline, if any.
1281    pipeline: Option<PipelineState>,
1282
1283    /// The bind group set at each index, if any.
1284    bind: ArrayVec<Option<BindState>, { hal::MAX_BIND_GROUPS }>,
1285
1286    /// The state of each vertex buffer slot.
1287    vertex: [Option<VertexState>; hal::MAX_VERTEX_BUFFERS],
1288
1289    /// The current index buffer, if one has been set. We flush this state
1290    /// before indexed draw commands.
1291    index: Option<IndexState>,
1292
1293    /// Dynamic offset values used by the cleaned-up command sequence.
1294    ///
1295    /// This becomes the final [`RenderBundle`]'s [`BasePass`]'s
1296    /// [`dynamic_offsets`] list.
1297    ///
1298    /// [`dynamic_offsets`]: BasePass::dynamic_offsets
1299    flat_dynamic_offsets: Vec<wgt::DynamicOffset>,
1300
1301    device: Arc<Device>,
1302    commands: Vec<ArcRenderCommand>,
1303    buffer_memory_init_actions: Vec<BufferInitTrackerAction>,
1304    texture_memory_init_actions: Vec<TextureInitTrackerAction>,
1305    next_dynamic_offset: usize,
1306}
1307
1308impl State {
1309    /// Return the current pipeline state. Return an error if none is set.
1310    fn pipeline(&self) -> Result<&PipelineState, RenderBundleErrorInner> {
1311        self.pipeline
1312            .as_ref()
1313            .ok_or(DrawError::MissingPipeline.into())
1314    }
1315
1316    /// Mark all non-empty bind group table entries from `index` onwards as dirty.
1317    fn invalidate_bind_group_from(&mut self, index: usize) {
1318        for contents in self.bind[index..].iter_mut().flatten() {
1319            contents.is_dirty = true;
1320        }
1321    }
1322
1323    fn set_bind_group(
1324        &mut self,
1325        slot: u32,
1326        bind_group: &Arc<BindGroup>,
1327        dynamic_offsets: Range<usize>,
1328    ) {
1329        // If this call wouldn't actually change this index's state, we can
1330        // return early.  (If there are dynamic offsets, the range will always
1331        // be different.)
1332        if dynamic_offsets.is_empty() {
1333            if let Some(ref contents) = self.bind[slot as usize] {
1334                if contents.bind_group.is_equal(bind_group) {
1335                    return;
1336                }
1337            }
1338        }
1339
1340        // Record the index's new state.
1341        self.bind[slot as usize] = Some(BindState {
1342            bind_group: bind_group.clone(),
1343            dynamic_offsets,
1344            is_dirty: true,
1345        });
1346
1347        // Once we've changed the bind group at a particular index, all
1348        // subsequent indices need to be rewritten.
1349        self.invalidate_bind_group_from(slot as usize + 1);
1350    }
1351
1352    /// Determine which bind group slots need to be re-set after a pipeline change.
1353    ///
1354    /// Given that we are switching from the current pipeline state to `new`,
1355    /// whose layout is `layout`, mark all the bind group slots that we need to
1356    /// emit new `SetBindGroup` commands for as dirty.
1357    ///
1358    /// According to `wgpu_hal`'s rules:
1359    ///
1360    /// - If the layout of any bind group slot changes, then that slot and
1361    ///   all following slots must have their bind groups re-established.
1362    ///
1363    /// - Changing the push constant ranges at all requires re-establishing
1364    ///   all bind groups.
1365    fn invalidate_bind_groups(&mut self, new: &PipelineState, layout: &PipelineLayout) {
1366        match self.pipeline {
1367            None => {
1368                // Establishing entirely new pipeline state.
1369                self.invalidate_bind_group_from(0);
1370            }
1371            Some(ref old) => {
1372                if old.pipeline.is_equal(&new.pipeline) {
1373                    // Everything is derived from the pipeline, so if the id has
1374                    // not changed, there's no need to consider anything else.
1375                    return;
1376                }
1377
1378                // Any push constant change invalidates all groups.
1379                if old.push_constant_ranges != new.push_constant_ranges {
1380                    self.invalidate_bind_group_from(0);
1381                } else {
1382                    let first_changed = self.bind.iter().zip(&layout.bind_group_layouts).position(
1383                        |(entry, layout)| match *entry {
1384                            Some(ref contents) => !contents.bind_group.layout.is_equal(layout),
1385                            None => false,
1386                        },
1387                    );
1388                    if let Some(slot) = first_changed {
1389                        self.invalidate_bind_group_from(slot);
1390                    }
1391                }
1392            }
1393        }
1394    }
1395
1396    /// Set the bundle's current index buffer and its associated parameters.
1397    fn set_index_buffer(
1398        &mut self,
1399        buffer: Arc<Buffer>,
1400        format: wgt::IndexFormat,
1401        range: Range<wgt::BufferAddress>,
1402    ) {
1403        match self.index {
1404            Some(ref current)
1405                if current.buffer.is_equal(&buffer)
1406                    && current.format == format
1407                    && current.range == range =>
1408            {
1409                return
1410            }
1411            _ => (),
1412        }
1413
1414        self.index = Some(IndexState {
1415            buffer,
1416            format,
1417            range,
1418            is_dirty: true,
1419        });
1420    }
1421
1422    /// Generate a `SetIndexBuffer` command to prepare for an indexed draw
1423    /// command, if needed.
1424    fn flush_index(&mut self) {
1425        let commands = self.index.as_mut().and_then(|index| index.flush());
1426        self.commands.extend(commands);
1427    }
1428
1429    fn flush_vertices(&mut self) {
1430        let commands = self
1431            .vertex
1432            .iter_mut()
1433            .enumerate()
1434            .flat_map(|(i, vs)| vs.as_mut().and_then(|vs| vs.flush(i as u32)));
1435        self.commands.extend(commands);
1436    }
1437
1438    /// Generate `SetBindGroup` commands for any bind groups that need to be updated.
1439    fn flush_binds(&mut self, used_bind_groups: usize, dynamic_offsets: &[wgt::DynamicOffset]) {
1440        // Append each dirty bind group's dynamic offsets to `flat_dynamic_offsets`.
1441        for contents in self.bind[..used_bind_groups].iter().flatten() {
1442            if contents.is_dirty {
1443                self.flat_dynamic_offsets
1444                    .extend_from_slice(&dynamic_offsets[contents.dynamic_offsets.clone()]);
1445            }
1446        }
1447
1448        // Then, generate `SetBindGroup` commands to update the dirty bind
1449        // groups. After this, all bind groups are clean.
1450        let commands = self.bind[..used_bind_groups]
1451            .iter_mut()
1452            .enumerate()
1453            .flat_map(|(i, entry)| {
1454                if let Some(ref mut contents) = *entry {
1455                    if contents.is_dirty {
1456                        contents.is_dirty = false;
1457                        let offsets = &contents.dynamic_offsets;
1458                        return Some(ArcRenderCommand::SetBindGroup {
1459                            index: i.try_into().unwrap(),
1460                            bind_group: Some(contents.bind_group.clone()),
1461                            num_dynamic_offsets: offsets.end - offsets.start,
1462                        });
1463                    }
1464                }
1465                None
1466            });
1467
1468        self.commands.extend(commands);
1469    }
1470
1471    fn vertex_buffer_sizes(&self) -> impl Iterator<Item = Option<wgt::BufferAddress>> + '_ {
1472        self.vertex
1473            .iter()
1474            .map(|vbs| vbs.as_ref().map(|vbs| vbs.range.end - vbs.range.start))
1475    }
1476}
1477
1478/// Error encountered when finishing recording a render bundle.
1479#[derive(Clone, Debug, Error)]
1480pub(super) enum RenderBundleErrorInner {
1481    #[error(transparent)]
1482    Device(#[from] DeviceError),
1483    #[error(transparent)]
1484    RenderCommand(RenderCommandError),
1485    #[error(transparent)]
1486    Draw(#[from] DrawError),
1487    #[error(transparent)]
1488    MissingDownlevelFlags(#[from] MissingDownlevelFlags),
1489    #[error(transparent)]
1490    Bind(#[from] BindError),
1491    #[error(transparent)]
1492    InvalidResource(#[from] InvalidResourceError),
1493}
1494
1495impl<T> From<T> for RenderBundleErrorInner
1496where
1497    T: Into<RenderCommandError>,
1498{
1499    fn from(t: T) -> Self {
1500        Self::RenderCommand(t.into())
1501    }
1502}
1503
1504/// Error encountered when finishing recording a render bundle.
1505#[derive(Clone, Debug, Error)]
1506#[error("{scope}")]
1507pub struct RenderBundleError {
1508    pub scope: PassErrorScope,
1509    #[source]
1510    inner: RenderBundleErrorInner,
1511}
1512
1513impl RenderBundleError {
1514    pub fn from_device_error(e: DeviceError) -> Self {
1515        Self {
1516            scope: PassErrorScope::Bundle,
1517            inner: e.into(),
1518        }
1519    }
1520}
1521
1522impl<T, E> MapPassErr<T, RenderBundleError> for Result<T, E>
1523where
1524    E: Into<RenderBundleErrorInner>,
1525{
1526    fn map_pass_err(self, scope: PassErrorScope) -> Result<T, RenderBundleError> {
1527        self.map_err(|inner| RenderBundleError {
1528            scope,
1529            inner: inner.into(),
1530        })
1531    }
1532}
1533
1534pub mod bundle_ffi {
1535    use super::{RenderBundleEncoder, RenderCommand};
1536    use crate::{id, RawString};
1537    use core::{convert::TryInto, slice};
1538    use wgt::{BufferAddress, BufferSize, DynamicOffset, IndexFormat};
1539
1540    /// # Safety
1541    ///
1542    /// This function is unsafe as there is no guarantee that the given pointer is
1543    /// valid for `offset_length` elements.
1544    pub unsafe fn wgpu_render_bundle_set_bind_group(
1545        bundle: &mut RenderBundleEncoder,
1546        index: u32,
1547        bind_group_id: Option<id::BindGroupId>,
1548        offsets: *const DynamicOffset,
1549        offset_length: usize,
1550    ) {
1551        let offsets = unsafe { slice::from_raw_parts(offsets, offset_length) };
1552
1553        let redundant = bundle.current_bind_groups.set_and_check_redundant(
1554            bind_group_id,
1555            index,
1556            &mut bundle.base.dynamic_offsets,
1557            offsets,
1558        );
1559
1560        if redundant {
1561            return;
1562        }
1563
1564        bundle.base.commands.push(RenderCommand::SetBindGroup {
1565            index,
1566            num_dynamic_offsets: offset_length,
1567            bind_group_id,
1568        });
1569    }
1570
1571    pub fn wgpu_render_bundle_set_pipeline(
1572        bundle: &mut RenderBundleEncoder,
1573        pipeline_id: id::RenderPipelineId,
1574    ) {
1575        if bundle.current_pipeline.set_and_check_redundant(pipeline_id) {
1576            return;
1577        }
1578
1579        bundle
1580            .base
1581            .commands
1582            .push(RenderCommand::SetPipeline(pipeline_id));
1583    }
1584
1585    pub fn wgpu_render_bundle_set_vertex_buffer(
1586        bundle: &mut RenderBundleEncoder,
1587        slot: u32,
1588        buffer_id: id::BufferId,
1589        offset: BufferAddress,
1590        size: Option<BufferSize>,
1591    ) {
1592        bundle.base.commands.push(RenderCommand::SetVertexBuffer {
1593            slot,
1594            buffer_id,
1595            offset,
1596            size,
1597        });
1598    }
1599
1600    pub fn wgpu_render_bundle_set_index_buffer(
1601        encoder: &mut RenderBundleEncoder,
1602        buffer: id::BufferId,
1603        index_format: IndexFormat,
1604        offset: BufferAddress,
1605        size: Option<BufferSize>,
1606    ) {
1607        encoder.set_index_buffer(buffer, index_format, offset, size);
1608    }
1609
1610    /// # Safety
1611    ///
1612    /// This function is unsafe as there is no guarantee that the given pointer is
1613    /// valid for `data` elements.
1614    pub unsafe fn wgpu_render_bundle_set_push_constants(
1615        pass: &mut RenderBundleEncoder,
1616        stages: wgt::ShaderStages,
1617        offset: u32,
1618        size_bytes: u32,
1619        data: *const u8,
1620    ) {
1621        assert_eq!(
1622            offset & (wgt::PUSH_CONSTANT_ALIGNMENT - 1),
1623            0,
1624            "Push constant offset must be aligned to 4 bytes."
1625        );
1626        assert_eq!(
1627            size_bytes & (wgt::PUSH_CONSTANT_ALIGNMENT - 1),
1628            0,
1629            "Push constant size must be aligned to 4 bytes."
1630        );
1631        let data_slice = unsafe { slice::from_raw_parts(data, size_bytes as usize) };
1632        let value_offset = pass.base.push_constant_data.len().try_into().expect(
1633            "Ran out of push constant space. Don't set 4gb of push constants per RenderBundle.",
1634        );
1635
1636        pass.base.push_constant_data.extend(
1637            data_slice
1638                .chunks_exact(wgt::PUSH_CONSTANT_ALIGNMENT as usize)
1639                .map(|arr| u32::from_ne_bytes([arr[0], arr[1], arr[2], arr[3]])),
1640        );
1641
1642        pass.base.commands.push(RenderCommand::SetPushConstant {
1643            stages,
1644            offset,
1645            size_bytes,
1646            values_offset: Some(value_offset),
1647        });
1648    }
1649
1650    pub fn wgpu_render_bundle_draw(
1651        bundle: &mut RenderBundleEncoder,
1652        vertex_count: u32,
1653        instance_count: u32,
1654        first_vertex: u32,
1655        first_instance: u32,
1656    ) {
1657        bundle.base.commands.push(RenderCommand::Draw {
1658            vertex_count,
1659            instance_count,
1660            first_vertex,
1661            first_instance,
1662        });
1663    }
1664
1665    pub fn wgpu_render_bundle_draw_indexed(
1666        bundle: &mut RenderBundleEncoder,
1667        index_count: u32,
1668        instance_count: u32,
1669        first_index: u32,
1670        base_vertex: i32,
1671        first_instance: u32,
1672    ) {
1673        bundle.base.commands.push(RenderCommand::DrawIndexed {
1674            index_count,
1675            instance_count,
1676            first_index,
1677            base_vertex,
1678            first_instance,
1679        });
1680    }
1681
1682    pub fn wgpu_render_bundle_draw_indirect(
1683        bundle: &mut RenderBundleEncoder,
1684        buffer_id: id::BufferId,
1685        offset: BufferAddress,
1686    ) {
1687        bundle.base.commands.push(RenderCommand::DrawIndirect {
1688            buffer_id,
1689            offset,
1690            count: 1,
1691            indexed: false,
1692        });
1693    }
1694
1695    pub fn wgpu_render_bundle_draw_indexed_indirect(
1696        bundle: &mut RenderBundleEncoder,
1697        buffer_id: id::BufferId,
1698        offset: BufferAddress,
1699    ) {
1700        bundle.base.commands.push(RenderCommand::DrawIndirect {
1701            buffer_id,
1702            offset,
1703            count: 1,
1704            indexed: true,
1705        });
1706    }
1707
1708    /// # Safety
1709    ///
1710    /// This function is unsafe as there is no guarantee that the given `label`
1711    /// is a valid null-terminated string.
1712    pub unsafe fn wgpu_render_bundle_push_debug_group(
1713        _bundle: &mut RenderBundleEncoder,
1714        _label: RawString,
1715    ) {
1716        //TODO
1717    }
1718
1719    pub fn wgpu_render_bundle_pop_debug_group(_bundle: &mut RenderBundleEncoder) {
1720        //TODO
1721    }
1722
1723    /// # Safety
1724    ///
1725    /// This function is unsafe as there is no guarantee that the given `label`
1726    /// is a valid null-terminated string.
1727    pub unsafe fn wgpu_render_bundle_insert_debug_marker(
1728        _bundle: &mut RenderBundleEncoder,
1729        _label: RawString,
1730    ) {
1731        //TODO
1732    }
1733}