Support for Oak 4 S and Oak 4 D models

[BarnabasTakacs]

Hi,

We have been using your cameras for a while in Unity with great success,
and recently purchased an Oak 4 S and D devices.
Are you planning to support those in Unity?

Thank you
Barnabas Takacs

[gespona]

Hi @BarnabasTakacs , Happy to hear your feedback using OAK with Unity. Feel free to share more details about your project and needs. It's very useful for us.
Regarding OAK4, I can confirm we're working to support those in the Unity plugin. Main rewriting is undergoing with the goal of using DepthAI v3 and common code base for all our cameras.
Right now, I can't provide any ETA but if you share more details about your project and needs, we will take into account during the release plan.

[gespona]

@BarnabasTakacs we're very interested in know more about your use case. Could you share more information? Thanks in advance.

[gespona]

@BarnabasTakacs just quick follow-up. We will start soon the migration to DepthAI v3 ... any feedback on your use case is valuable for us to decide priorities. What AI models are you running? What is your use case? Thanks in advance.

[BarnabasTakacs]

Hello, sorry for the delayed response. We use the cameras in a variety of ways, primarily to drive visuals and particle systems for real-time graphics, but also in medical and agro-tech projects to process the captured images for object detection, 3D pose estimation, keypoint recognition, tracking etc.

Unity is our primary choice for developing many different programs, interfaces, so it would be a key element to be able to use that (instead of Python libraries).

One of the projects I am looking to solve in particular, requires us to train keypoints of animals. Similar to DeepLab cut model, but I think in this case it would be YoloV8 running on your device already (I am open to suggestions for other models if you think they perform better on your architecture). Our goal here would be to run all recognition and measurements on the camera withou the
need for a second mini-computer on location.

I hope this helps, but let me know if you have more questions.

[gespona]

@BarnabasTakacs Thanks for sharing the details!
If you’re working with animal key-points, you might find this example helpful:
https://github.com/luxonis/depthai-experiments/tree/gen3/neural-networks/pose-estimation/animal-pose

Just to confirm my understanding:
• You’d like to run all ML models on the camera itself to avoid using a separate mini-computer, but you’ll still run the Unity application elsewhere—correct?
• Or are you considering Unity’s IoT/Industrial license?

Let me know if I’ve got that right or if you’re exploring a different setup!

[BarnabasTakacs]

Hi, sorry for the late response.

Thank you for the animal example, I tested it already. Works great.

Regarding the Unity interface. We use Unity to process and visualize a combination of raw data and AI-processed images.
That ranges from capturing the camera's image (just as if it was a webcam) and run AI models on it within Unity, as well
as combining the on-camera processing (like depth-based particle systems or masking 3D objects etc). We have a wide range of applications.

Unity also gives us the advantage of being able to deploy the final system as a distributable/installable exe.

[BarnabasTakacs]

Regarding the license we use Unity Pro for the Apple Vision Pro, but not the Industrial license.
For the OAK applications I think it would be preferential to use the plane 'vanilla' license for everybody to be able to use it.

[gespona]

Regarding the license we use Unity Pro for the Apple Vision Pro, but not the Industrial license. For the OAK applications I think it would be preferential to use the plane 'vanilla' license for everybody to be able to use it.

Totally agree — that was the original idea when I started the plugin, and it remains the main approach as we move forward with OAK 4.
That said, it’s always a potential discussion point if someone wants to push further "edge" use cases 😉

Thanks so much for all your feedback. I’ll keep this ticket open so others can continue to share their use cases.
We also have the main roadmap issue, where I’ll keep posting updates — hopefully more news coming soon!