Add AlternativeFlatten

[johnynek]

AlternativeFlatten gives us mapFlatten with a natural functor like law. Here is an example:

fa.mapFlatten { a => List(a, a) }.mapFlatten { b => Some(b) } == fa.mapFlatten { a => List(a, a).map(Some(_)) }

This is useful to model the kind of "flatMap" scala collections have, where we are not really dealing with Monadic flatMap. Also this is very useful for distributed systems such as spark and scalding, which offer this kind of abstraction but the core data type is not a monad (RDD and TypedPipe respectively).

I was talking about something like this, and @non suggested we use G[_]: Foldable as the inner type, upon trying to formalize the laws, I moved to G[_]: Traverse which seems to satisfy all use cases and allows us to state some nice laws.

[codecov-io]

Codecov Report

Merging #1337 into master will increase coverage by 0.08%.
The diff coverage is 94.23%.

@@            Coverage Diff             @@
##           master    #1337      +/-   ##
==========================================
+ Coverage   91.39%   91.47%   +0.08%     
==========================================
  Files         237      241       +4     
  Lines        3567     3615      +48     
  Branches       64      120      +56     
==========================================
+ Hits         3260     3307      +47     
- Misses        307      308       +1

Impacted Files	Coverage Δ
...rc/main/scala/cats/syntax/alternativeFlatten.scala	0% <0%> (ø)
...cats/laws/discipline/AlternativeFlattenTests.scala	100% <100%> (ø)
core/src/main/scala/cats/instances/stream.scala	95.74% <100%> (+2.72%)	⬆️
.../main/scala/cats/laws/AlternativeFlattenLaws.scala	100% <100%> (ø)
core/src/main/scala/cats/AlternativeFlatten.scala	100% <100%> (ø)
core/src/main/scala/cats/instances/list.scala	100% <100%> (ø)	⬆️
core/src/main/scala/cats/instances/vector.scala	97.87% <100%> (+0.5%)	⬆️
core/src/main/scala/cats/Functor.scala	52% <0%> (-48%)	⬇️
core/src/main/scala/cats/functor/Profunctor.scala	60% <0%> (-40%)	⬇️
core/src/main/scala/cats/arrow/Split.scala	66.66% <0%> (-33.34%)	⬇️
... and 13 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 41d9ef8...8f9315c. Read the comment docs.

[kcsongor]

I don't think this makes sense as a separate type-class.
I would love for someone to show me how FunctorFilter can be implemented for Functors that aren't equpped with a monoidal property, (i.e. that aren't Applicative).
In case it can't, then I think this is duplicating what Traverse does?

[kcsongor]

actually not applicative, but something with a f a -> f a -> f a

[johnynek]

I'm a bit confused. Can you show a proof that this is equivalent to
Traverse? I don't see it.

As I said, scalding and spark (not to mention the scala standard library's
flatMap) are all really this mapFlatten operation I'm proposing.

In the absence of a proof of equivalence, I'd say it is clearly general.
On Mon, Aug 29, 2016 at 09:36 Csongor Kiss notifications@github.com wrote:

actually not applicative, but something with a f a -> f a -> f a

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#1337 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAEJdsoJPS4hpcCWoJUkfOJhaLNSKPgZks5qkzSlgaJpZM4Ju0KK
.

[ceedubs]

I could be wrong about this, but after taking a quick glance it seems to me that if we implemented an instance of this type class that only ever did anything with the first item in the provided G (and therefore we could create an instance for Option) it would satisfy the laws, but to me this seems like it would violate unwritten assumptions about what this type class represents. Is there a way that these assumptions can be codified as laws? I'm not sure if there's a way you could unless you coupled this with something like Unfoldable.

[johnynek]

I think if you have MonoidK and Applicative on F[_] you can make a law like:

pure(a).mapFlatten(f) == f(a).foldRight(empty) { (b, end) =>
combineK(pure(b), end) }

On Mon, Aug 29, 2016 at 09:58 Cody Allen notifications@github.com wrote:

I could be wrong about this, but after taking a quick glance it seems to
me that if we implemented an instance of this type class that only ever did
anything with the first item in the provided G (and therefore we could
create an instance for Option) it would satisfy the laws, but to me this
seems like it would violate unwritten assumptions about what this type
class represents. Is there a way that these assumptions can be codified as
laws? I'm not sure if there's a way you could unless you coupled this with
something like Unfoldable #1132.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#1337 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAEJdpVaARjUZVg6wgGcDN3TsmENvB6iks5qkznNgaJpZM4Ju0KK
.

[kcsongor]

@johnynek sorry for the delay, I have just gotten around to writing down a few thoughts I've had on this: #1365
However, I don't have a full proof, it is difficult to prove the 'necessary' part. I think the question really is whether it is possible to remove an element from a structure that doesn't have a foldable instance (by doesn't have, I mean it cannot admit a lawful instance for some reason).
I'll try and come up with a proof for algebraic data types, and then think about the general case, or prove myself wrong.

[non]

@kcsongor So to help motivate the type class, consider TypedPipe[A].

This type is a bit like a Task[A] but also like a collection. It represents operations on zero-or-more rows of data in a distributed filesystem which you can't easily access, and which won't be run until the end of the world.

It definitely has a Functor instance (it has a standard map method). Since it also has a pure method and the ability to create products (in its case Cartesian products), its instance is Applicative. It also has universal concatenation and empty methods, so it has a MonoidK instance as well, making the whole structure an Alternative instance.

However it doesn't a have true flatMap method (i.e. A => TypedPipe[B]); instead it supports something like A => Iterable[B] (or A => G[FB] for Foldable[G]). You can't actually extract A values out of TypedPipe[A], so it itself is not foldable.

It could be that there aren't any FunctorFilter instances which aren't instances of this more powerful type class. I agree with your reasoning that FunctorFilter (and this type class) should require empty since you can derive it using .filter(_ => false). I can't easily think of types where you can contract (but not expand) their structure.

However I think it's worth considering this type class on its own merits. Once you have a full-blown Monad you give up a lot of control around its execution (compared to an applicative structure). Whether we call this FunctorFlatten or instead create some kind of souped-up version of Applicative or Alternative I feel like this would be a useful addition.

[kcsongor]

@non Thanks for that example, it was very useful!

The assumption I made was that the intermediate values need to be of the same type as the container, but that assumption was too restrictive, since it is enough that we can produce some intermediate values of type G[B], which can be collapsed into a single G[B], which can then be turned back into an F[B].
So, as long as there is a natural transformation G ~> F, I can weaken the restriction from flatMap[A, B](fa: F[A])(afb: A => F[B]) => F[B] to flatMap2[A, B, G[_]: Foldable](fa: F[A])(afb: A => G[B]) => F[B]
This is still coming from FunctorFilter, but flatMap2 now looks exactly like mapFlatten from FunctorFlatten!

I understand that this is useful from a practical perspective, when you can write a more efficient way of filtering, and don't want to use some almost incidental filter function that is derived from other classes.

[andyscott]

From a learning perspective, someone who is familiar with Scala and Scala's flatMap may find the principled flatMap irritating or too rigid to work with. Providing a mapFlatten or something similar (that's easily accessible) would ease the adoption process. I would find it convenient in my own work as well.

[johnynek]

I'll try to rework this, perhaps based on Alternative (all the examples I
know of are also Alternative).
On Sat, Sep 17, 2016 at 07:01 Andy notifications@github.com wrote:

From a learning perspective, someone who is familiar with Scala and
Scala's flatMap may find the principled flatMap irritating or too rigid to
work with. Providing a mapFlatten or something similar (that's easily
accessible) would ease the adoption process. I would find it convenient in
my own work as well.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1337 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAEJdh9uD6XUvAYVwSBYDsu6iR-19NNMks5qrBzbgaJpZM4Ju0KK
.

[adelbertc]

As a side note, such existence of these type classes that branch off the "main" (I don't like that adjective since it implies theres a certain blessed type class hierarchy but I can't think of a better word right now) type class hierarchy can cause issues. For instance, if you have

def foo[F[_]: FunctorFlatten: Applicative] = ...

If you try to use anything that requires an implicit Functor[F] you will get ambiguous implicits. This is a manifestation of #1210 - more discussion here in this PR here: #1379

[johnynek]

@ceedubs what do you think of the new laws? They seem legit to me?

[adelbertc]

Please see #1379 (comment) I would like to make a decision on that before adding more of such type classes like this.

[kailuowang]

@johnynek are we still interested in this?

[sellout]

This should be in cats-mtl now, no? (Considering it extends classes that I think have moved to cats-mtl. Also, this should be a superclass of MonadCombine, maybe? (or whatever the equivalent in cats-mtl is now – I’ve fallen behind in this stuff)

[johnynek]

@sellout the examples I care most about are not Monads.

[sellout]

@johnynek Right – I’m just wondering where this class fits in the hierarchy – is it between Alternative and MonadCombine, or is it a separate branch off Alternative? It seems like the former to me, since the fundamental flatten operation works with either (AlternativeFlatten m, Traversable f) => m (f a) -> m a or (MonadCombine m, Foldable f) => m (f a) -> m a, but that’s just a hunch.

[kailuowang]

Closing stale PRs. Feel free to reopen if there is interest to revive the effort.

[johnynek]

Note, SelectiveZero seems to be equivalent to this when G=Option:

https://github.com/snowleopard/ideas/blob/7692eba70758a48cf2dc9224627df906fe078a9b/src/SelectiveZero.hs#L41