should on_message cloud function be a child or a sibling of queue

[ekeren]

We have the following code in our examples:

bring cloud;

let queue = new cloud.Queue();
let bucket = new cloud.Bucket();

queue.on_message((m: str): str ~> {
  bucket.put(“wing.txt”, “Hello ${m}!”)
});

The structure that it currently produces in the console is:

Root
|- Bucket
|- Queue
    |- onMessage-62363464

Meaning that the cloud function that saves the file on the bucket is a child of the queue.
I think that it is more correct that a queue is just a leaf, and doesn’t own the cloud function, I believe that the proper representation of this hierarchy should be:

Root
|- Queue
|- Bucket
|- Function 

Which means that the on_message function should not accept an inflight function, it should accept a cloud.Function
and the queue on_message property is set to that cloud.Function but the queue itself is not the functions' parent

A more descriptive way to write the above code will be:

bring cloud

let queue = new cloud.Queue();
let bucket = new cloud.Bucket() be store-greeting;
let handler = new cloud.Function((m: str): str ~> {
  bucket.put(“${m}.txt”, “Hello ${m}!”)
}) be "execute-greetings";

queue.on_message(handler);

And the corresponding hierarchy will be:

Root
|- Queue
|- store-greeting
|- execute-greetings 

The queue itself is just an entry point to this greeting system, it should not contain the actual logic.

[staycoolcall911]

@ekeren - the last line of the 2nd code sample should be queue.on_message(handler);, right?

[ekeren]

sure, updated. Thanks

[Chriscbr]

If I understand correctly, there's two issues being discussed - one is whether any function resource added/attached by on_message should be a sibling or child of the queue, and the other is whether the method should accept an inflight or a function resource.

The first issue is pretty straight forward - I totally agree, currently the way the resource is generated as a child of the queue doesn't totally make sense. Luckily this is a pretty easy fix. I'll write up a PR to update this.

The second issue is a bit more tricky. In an early iteration of the Queue resource, we actually did have the on_message method accept a cloud.Function resource like you described. But I found some technical issues with this model and compiling it to the cloud. I'm happy to give more details, but it will probably take a bit more time to write up.

TL;DR - subscribing to the events of a queue or other resource usually means there needs to exist code somewhere to convert from a cloud provider's native event structures (like SQSEvent) to the Wing SDK types. The most straightforward way to do this is to add a wrapper to the user's code to preprocess inputs, but this would require modifying user's code, and changing the effective type of the user's resource from cloud.Function<input: str> to cloud.Function<input: SQSEvent>. To avoid this surprising change, the current API instead expects an inflight with input: str as a parameter, and generates the cloud.Function<input: SQSEvent> for you. There are other ways we could implement this without adding wrappers to user code (for example, performing data transformations outside of user code, like in an additional lambda function), but these solutions might have performance costs.

[ekeren]

Another example of the responsibility constraints is what happens if I need to I state that the runtime require nodejs:18.*

This kind of requirement should be a cloud.Function requirement and is the concern of the person passing the handler to on_message I would not expect for the queue to fulfill this requirement

[Chriscbr]

That's a valid concern - another example of this is if someone wants to add environment variables to their cloud function. Right now we support this through an additional props argument, I believe:

queue.on_message(
  (event) ~> { print(event) },
  env: { DEBUG: "true" }
);

[eladb]

BTW @ekeren for pure wing code, users shouldn't be able to specify the node runtime on functions. The fact that this compiles to JS and runs in node is an technically implementation details

[ekeren]

One reason for having to rely on a specific runtime is the fact that you bring a package that requires a specific node version or higher

[Chriscbr]

I think the version of the node/python/etc. runtime you're using might blur into more of an operational concern

[eladb]

I think we need both options: one that you to "bring your own worker" (queue.add_worker()), and one that is a ergonomic sugar that reduces boilerplate (queue.on_message()).

I am not sure about the APIs, we need to discuss in a wider context as part of the SDK RFC.

As for resource scoping for the implicit cloud.Function (the topic of this discussion), I can see how it might be friendlier to put the implicitly created function as a sibling of the queue instead of its child. I am okay with that if we feel this makes more sense for users. I love it that Wing Console surfaces these questions :-).

P.S. We should have a deeper discussion on our event-driven API pattern as part of a Wing SDK RFC. @Chriscbr I know that @staycoolcall911 and you discussed kicking off this RFC, and I think we are ready to write it, and that it will really help bring up all these topics with a holistic view of the entire surface area.

[ekeren]

Don't you feel that there is a correlation between the identity of the "creator" of the cloud.Function resource and the hierarchy (sibling vs parent)

It seems unintuitive to me that I can create a sibling resource

[eladb]

I think it's okay to think about this as an aesthetic/ergonomic aspect of how the tree "looks" and how intuitive it is to navigate it rather than a puristic concern of maintaining strict ownership/scoping relationship. As long as the cloud.Function ID is derived from the queue (e.g. ${queue.id}-on_message), then it would be okay to make it a sibling instead of a child.

[eladb]

Let's fold this discussion into #858 ?