Adapter find options additions

[kketch]

I'm wondering how the Adapter.find API could evolve if fortune was to support more querying features

Available options currently are :

• match
• exists
• range

The API does not seem to allow more advanced queries with negation for match, or a logical or operator.
An example is the waterline query language which has support for some features that I think would be useful in fortune :

• not
• or
• startWith
• contains
• endsWith

I'm not sure how such features would change the current API for adapters. I know that implementing this on fortune and adapters could be a lot of work. But I'm interested to work on these options

What about adding these new features ?

[gr0uch]

I don't know, I don't want to go down the road of adding every possible querying option, because then there would be too much, and not every database supports all the same query options. match and exists were obvious, and range was added due to the sort option. Also the current querying options work naturally on all types, like Date, Buffer, Number, etc. I can see not and or being useful, but not sure if others can be used generically. Also I'm not sure about the performance and use cases of these options, i.e. negation operators are generally slow, and very few user interfaces actually use negations or or operators. It would also raise the bar for writing an adapter, and my idea is that it should be kept simple, for the most common use cases.

The alternative is to write adapter-specific options. Each database has its own strengths and weaknesses, which makes it not really desirable to standardize all possible querying options, but to just provide a common subset. For example if it's decided that your application will use MongoDB, you would just use the query option provided by the adapter for MongoDB-specific querying.

[kketch]

With adapter specific options there is the risk that multiple adapters implement similar options but with different APIs.
Adapters now have the query option where some expect it to be a function. Sadly for these adapters this query option is not available over the wire with fortune.net.request.
But yes I think you're right there isn't many use case in web apps for or or not.
In fact starts, ends & contains are already available on some adapters, nedb and mongodb since regexes can be passed in match and even over the wire thanks to msgpack support for encoding and decoding them.

But of course I cannot use regexes for other adapters like pqsql.
And this three options are for pattern matching and may only work on strings, so they cannot indeed be used generically on all types.
Maybe some support from adapters for more advanced querying including over the wire is what i'm looking for here. I think I will investigate how this could be supported outside of fortune

[gr0uch]

After thinking about implementing it, I think that not would be fairly simple. The tricky part is or, and, maybe this could simply be copied from MongoDB's syntax, but I'm not sure how much this will complicate things.

[kketch]

It will complicate things surely, especially for the memory, redis & indexedDB adapters. But I think that it will also raise fortune as a strong alternative to other data / ORM librairies / frameworks.

I found out that these features are rather unpleasant/unpratical to implement outside of fortune's core and adapters.

Would you also consider starts, contains and ends for strings ? I'm not very objective on these since it supports my use case for fortune, but it seems to be a fairly standard feature as not, or, and are.

Regarding the syntax, I thought of MongoDB's syntax too, but it would introduce great changes to the current query API, since the current query options match, range, exists could be nested multiple times under or objects for instance

[gr0uch]

For strings, why not just Regular Expressions?

Currently it already works if you pass a RegExp to match for NeDB and MongoDB, I guess this would be a compatible change. PostgreSQL also has support for querying by RegExp.

I think and/or can be added incrementally without breaking the existing functionality, in a similar style as MongoDB.

[kketch]

I'm fine with regular expressions. It shouldn't be hard to support them in the memory, redis & indexedDB adapters. Indeed NeDB & MongoDB adapters already support them. However regarding postgres I'm not sure if javascript regular expressions are compatible with it