Isaac Sim¶

Runs inside Isaac Sim (see Simulators). Importing twinrobo.plugins.isaac works without Isaac; its classes need a running SimulationApp.

IsaacCameraTwin ¶

IsaacCameraTwin(twin: CameraTwin, camera: str | None = None, match_fov: bool = True, resolution: tuple[int, int] | None = None, device: str | device | None = None, rectify: bool = False, rt_subframes: int = 1, near_clip_m: float = 0.01, render: str = 'psf', rays_per_pixel: int = 8, pupil_views: int = 7, shading: str = 'corrected', view_oversample: float = 1.0)

A CameraTwin rendering from an Isaac Sim (USD) camera prim.

Parameters:

Name	Type	Description	Default
`twin`	`CameraTwin`	The CameraTwin (`CameraTwin.from_spec` / `from_catalog`).	required
`camera`	`str \| None`	Path of a `UsdGeom.Camera` prim (or call `attach` later).	`None`
`match_fov`	`bool`	Render with the twin's FoV. `False` keeps the camera's own lens.	`True`
`resolution`	`tuple[int, int] \| None`	`(W, H)` render size; only for twins without intrinsics.	`None`
`device`	`str \| device \| None`	Torch device for the CameraTwin pipeline (default: the PSF bank's).	`None`
`rectify`	`bool`	Undistort the output like vendor SDKs (see `CameraTwin.process`).	`False`
`rt_subframes`	`int`	RTX subframes per rendered frame (more: less ghosting after large motions, slower).	`1`
`near_clip_m`	`float`	Near clipping distance in meters. USD's default near clip is 1 scene unit (1 m on a meter stage), which would hide what a robot camera sees up close, so the view uses this instead of the camera's.	`0.01`
`render`	`str`	`"psf"` or `"raycast"` (see the module docs).	`'psf'`
`rays_per_pixel`	`int`	Rays traced per pixel and wavelength (ray cast).	`8`
`pupil_views`	`int`	Shading views rendered across the lens' entrance pupil (ray cast).	`7`
`shading`	`str`	`"corrected"` (ISP lens-shading correction) or `"raw"` (sensor vignetting and cos^4 falloff kept); ray cast.	`'corrected'`
`view_oversample`	`float`	Resolution of the pupil views relative to the sensor; above 1 resolves detail thinner than a pixel (ray cast).	`1.0`

attach ¶

attach(camera: str) -> None

Render from USD camera camera (a prim path).

close ¶

close() -> None

Remove the render product, annotators and the view prim.

render_ideal ¶

render_ideal(step: bool = True) -> tuple[Tensor, Tensor]

The pinhole view: (rgb linear [1, 3, H, W], depth m [1, 1, H, W]), upright.

step renders a new frame first (omni.replicator.core.orchestrator.step, without advancing the timeline). Pass False if your loop already rendered this frame, e.g. with world.step(render=True).

get_frame ¶

get_frame(timestamp: float | None = None, step: bool = True, force_depth: bool = False) -> CameraFrame

Render and process one frame. frame.rgb is linear [1, 3, H, W] on device.

frame.depth is readable only if the camera's depth output is simulated (outputs.depth in its spec) or with force_depth=True (simulator ground truth).

lens ¶

Lens-ray rendering in Isaac Sim: a USD camera as a lens scene.

The ray-cast method ("raycast") is simulator independent (twinrobo.optics.lensrender); this module supplies Isaac's side:

shading views across the pupil: child cameras of the camera prim (TwinRoboPupil<i>), each shifted in the camera's own frame to a pupil cell center, with the views' wide FoV, each with a Replicator render product (rgb + distance_to_image_plane). They render in the same Replicator step as the main view.
camera pose: the camera prim's world transform (stage units -> meters).
scene triangles: every visible gprim of the stage (meshes, cubes, spheres, cylinders, capsules, cones, planes, including instance proxies), posed per frame from USD transforms into one Warp BVH.

Limits: poses are read from USD, so a simulation must write them there (Isaac's default for scripted and physics prims); deforming meshes keep their first shape; point instancers are not ray cast.

UsdSceneMesh ¶

UsdSceneMesh(stage, device)

Bases: TriangleMesh

Every visible gprim of a USD stage as ray-cast triangles, posed per frame from USD.

IsaacLensScene ¶

IsaacLensScene(camera: str, views: PupilViews, near_clip_m: float = 0.01)

A USD camera as a lens scene (twinrobo.optics.lensrender.LensRayRenderer.render_scene).

Creates the pupil-view cameras and their render products under camera; close removes them. The caller renders (one Replicator step renders the main view and every pupil view) before LensRayRenderer.render_scene.

ready ¶

ready() -> bool

Whether every pupil view has rendered a frame.

set_pinhole ¶

set_pinhole(cam, fx_px: float, width: int, height: int) -> None

Give a UsdGeom.Camera a centered pinhole: fx_px pixels at width x height.

prim_triangles ¶

prim_triangles(prim) -> tuple[ndarray, ndarray] | None

Local-frame triangles of a USD gprim (as RTX draws it), or None if unsupported.