Archive
On faith, religion, conjectures and Schubert calculus
Just in time for the holidays, Colleen Robichaux and I wrote this paper on positivity of Schubert coefficients. This paper is unlike any other paper I had written, both in the content and the way we obtained the results. To me, writing it was a religious experience. No, no, the paper is still mathematical, it’s just, uhm, different. Read this post till the end to find out how, or go straight to the paper (or perhaps both!)
Faith, religion and miracles — math got them all!
Mathematicians rarely if ever discuss the subject. When they do, they get either apologetic about it “there are no contradictions…”, or completely detached as if you can be deeply religious on Sunday morning and completely secular on Tuesday afternoon. See e.g. Robert Aumann’s extensive interview or this short clip by Freeman Dyson, two of the most admired people in their respective fields.
I have neither a religious education nor a specialized knowledge to speak on the subject, obviously. But in the age of Twitter (uhm, X.com, sorry) that minor issue never stopped anyone. So having said that, let me give a very vague and far-fetching primer to mathematicians in the language you could relate. This might prove helpful later on.
Faith is the foundation of it all. You really can’t do math without foundations. Outside of certain areas you don’t really need to understand it or even think about it all that that much. Just accept it and you’ll better off. For example, it is likely that the more you think about consistency of PA the less certain you get about what you are doing, so stay on the happy side.
This does not mean you need to be consistent with your owb tenants of faith. For example, it’s perfectly fine to have a paper in algebra using the Axiom of Choice (AC) while in another in descriptive set theory, where you go out of your way to avoid AC, and that’s the whole point of the paper. Still, it’s not like you were ever doubting AC — more like you have a multifaith personality.
Belief system is what it sounds like. If you are a working mathematician you probably already have a lot of opinions on a wide range of research questions, conjectures, etc. For example, maybe you accept some conjectures like Riemann Hypothesis, reject others like Navier-Stokes, remain very interested but undecided on problems like P vs. NP, and couldn’t care less about others like Yang-Mills. It’s all well and good, obviously.
There are many more shades of gray here. For example, you might believe in the abc conjecture, think that it hasn’t been proved yet, but willing to use it to prove other results whatever the status. Or perhaps, you believe that Goldbach’s conjecture is definitely true, that it’s a matter of time until it’s proved, but are completely unwilling to use it as an assumption (I used it as an assumption once, maddening some old school referees; luckily the odd version of GC holds and was sufficient). Or perhaps you are aware that the original proof of the Lyusternik-Schnirelmann theorem on three closed geodesics had major issues, as were many subsequent proofs of the theorem; still you are willing to trust Wikipedia that the theorem is true because you don’t care all that much anyway.
Ideally, you should question your beliefs whenever possible. If you are thinking about a conjecture, are you sure you believe in it? Maybe you should try to disprove it instead? It’s ok to change your mind, to try both directions, to believe or even to disbelieve all authorities on the matter. I have written extensively about the issue in this blog post, so let’s not rehash it.
One more thing about a belief system is that different beliefs usually have different levels of confidence. Some beliefs are core and people rarely change their mind. These don’t fall into faith category, in a sense that different people can have different core beliefs, sometimes in contradiction with each other. This is usually why a vast majority might strongly believe some conjecture is true, while a vocal minority might believe it’s false just as strongly.
For example, some people believe that mathematical order is absolutely fundamental, and tend to believe that various structures bring such an order. My core belief is sort of the opposite — because of whatever childhood trauma I experienced, I believe in universality theorems (sometimes phrased as Murphy’s law), that things can be wildly complicated to the extreme unless there is a good reason for them not to. Mnëv’s universality theorem is perhaps the most famous example, but there are many many others. This is why I disprove conjectures often enough and prove many NP– and #P-completeness results — these are different manifestations of the same phenomenon.
Religion is what you do with your belief system, as in practicing religion. If you have a lot of beliefs that doesn’t make you smart. That makes you opinionated. To be considered smart you need to act on your beliefs and actually prove something. To be considered wise you need the ability to learn to adjust your belief system to avoid contradictions with your other beliefs as new evidence emerges, and make choices that lead somewhere useful.
In mathematics, to practice an organized religion is to be professional, when you get paid for doing research. The process is fundamentally communal involving many people playing different roles (cf. this Thurston’s MO answer). Beside the obvious — researcher, editor, referee, publisher — there are many others. These include colleagues inviting you to give talks, departmental committees promoting you based on letter written by your letter writers. Some graduate students will be studying and giving talks on your work, others will be trying to simplify the arguments and extend them further. The list goes on.
In summary, it is absolutely ok to be an amateur and have your own religious practice. But within a community of like-minded scholars is where your research becomes truly valuable.
Miracles are the most delightful things that ever happen to you when learning or when doing mathematical research. It’s when you discover somethings, perhaps even prove that it works, but remain mystified as to why? What exactly is going on that made this miracle happen? Over time, you might learn one or several reasons, giving you a good explanation after the fact. But you have to remember your first impression when you just learned about the miracle.
It’s even harder to see and acknowledge miracles in celebrated textbook results. You have to train yourself to see the miracles for what they are, rather than for what they now appear to be when textbook packages them in neat nice boxes with a bow on top. One way to remind yourself of miracle powers is to read “Yes, Virginia“ column that I mentioned in this holiday post — it will melt you heart! Another way is to teach your favorites — when you see a joyful surprise in your students’ eyes, you know you just conveyed a miracle!
This may depend on your specific area, but in bijective combinatorics you have to believe in miracles! Otherwise you can never fully appreciate prior work, and can never let yourself loose enough to discover new miracles. To give just one example, the RSK correspondence is definitely on everyone’s the top ten list of miracles in the area. By now there are at least half a dozen ways to understand and explain it, but I still consider RSK to be a miracle.
Of course, one should not get overexcited about every miracle they see and learn to look deeper. For example, a combinatorial interpretation of Littlewood–Richardson coefficients is definitely a miracle, no doubt about it. But after some meditation you may realize that it’s really the same miracle as RSK (see §11.4 in my OPAC survey).
Backstory of the bunkbed conjecture paper
After I wrote this blog post about our disproof of the Bunkbed Conjecture (BBC), the paper became viral for 15 minutes. Soon after, I received several emails and phone calls from journalists (see links in the P.P.S. of that post). They followed the links to my earlier blog post about disproofs of conjectures and asked me questions like “What is the next conjecture do you plan to disprove?” and “Do you think disproving conjectures is more important than proving them?” Ugh… 😒
While that earlier post was written in a contrarian style, that was largely for entertainment purposes, and not how I actually think about math. Rather, I have an extensive somewhat idiosyncratic belief system that sometimes leads me think that certain conjectures are false. But breaking with conventional wisdom is a long and occasionally painful process. Worse, being a contrarian gives you a bad rap as you are often get confused with being nihilistic.
So let me describe how I came to believe that BBC is false. I was asked this multiple times, always declining out of fear of being misunderstood and misquoted. but it’s a story worth telling.
This all started with my long FOCS paper (joint with Christian Ikenmeyer), where we systematically studied polynomial inequalities and developed a rather advanced technology to resolve questions like “which of these can be proved combinatorially by a direct injection?” To give a basic example, if the defect (difference between two sides) is a polynomial that is an exact square, then the polynomial is obviously nonnegative but it often can be shown that the defect has no combinatorial interpretation, i.e. not in #P. See more in this blog post on this.
Now, I first learned about the Bunkbed Conjecture soon after coming to UCLA about 15 years ago. Tom Liggett who was incredibly kind to me, mentioned it several times over the years, always remarking that I am “the right person” to prove it. Unfortunately, Tom died just four years ago, and I keep wondering what he would have made of the story…
Anyway, when writing my OPAC survey two years ago, I was thinking about the problem again in connection to combinatorial interpretations, since BBC becomes a polynomial inequality when all edge probabilities are independent variables. Like everyone else, I assumed that BBC is true. But I figured that the counting version is not in #P since otherwise a combinatorial proof would have been found already (since many strong people have tried). So I made this into Conjecture 5.5 in the OPAC paper, and suggested it to my PhD student Nikita Gladkov.
I believed at that time that there must be a relatively small graph on which the BBC defect will be a square of some polynomial, or at least some positive polynomial (on a [0,1]n hypercube of n variables) with negative coefficients. That was our experience in the FOCS paper. Unfortunately, this guess was wrong. In our numerous experiments, the polynomials in the defect seemed to have positive coefficients without an obvious pattern. It was clear that having a direct injective proof would have been a miracle, the kind of miracle that one shouldn’t expect in the generality of all graphs.
This led to a belief contradiction — either a stronger version of BBC holds for a noncombinatorial reason, or BBC is false. In the language above, I had a core belief in the power of combinatorial injections when there are no clear obstructions. On the other hand, I had only a vague intuition that BBC should hold because it’s the most natural thing and because if true it would bring a bit of order to the universe. So I changed my mind about BBC and we started looking for a counterexample.
Over the next two years I asked about BBC to everyone I met, suggesting that it might be false in hope someone, anyone, gives a hint on how to proceed and what to rule out. Among those who knew and had an opinion about the problem, everyone was sure it’s true. Except for Jeff Kahn who lowered his voice and very quietly told me that he also thinks it’s false, but made me a promise not to tell anyone (I hope it’s ok now?) I think he was hinting I shouldn’t say these things out loud to avoid getting the crank reputation. I didn’t listen, obviously. Not being in the area helped — nobody took me seriously anyway.
In the meantime, Nikita and his friend Alexander Zimin made quite a bit of effort to understand multiple correlation inequalities (FKG style) for percolation on general graphs. This eventually led to the disproof as explained in the BBC blog post mentioned above.
Schubert positivity
In Algebraic Combinatorics, Schubert coefficients are nonnegative integers which generalize the Littlewood-Richardson (LR) coefficients mentioned above. Since the latter have extremely well studied combinatorial interpretations, the early hope was that Schubert coefficients would also have one. After decades of effort and advances in a handful of special cases, this became a major open problem in the area, the subject of numerous talks and papers.
I never believed this hope, not for a second. In the OPAC survey, I stated this as a conjecture: “Schubert coefficients are not in #P” (see Conj. 10.1). Again, this not because I was a contrarian — I had my reasons, three in fact.
First, that’s because I studied the miracle of RSK and the related miracle of LR-coefficients for over thirty years (yes, I am that old!) As a bijection, RSK is extremely rigid. So if any of the (essentially equivalent) combinatorial interpretations of LR-coefficients could generalize directly, it would have been done already. However, the progress has been exceedingly difficult (see e.g. Knutson’s 2022 ICM paper).
Second, I also have a strong belief that miracles are rare and don’t happen to the same combinatorial objects twice for different reasons. This is a variation on “lightening doesn’t strike twice” idea. It is in principle possible that LR-coefficients have a completely new combinatorial interpretation radically different from the 20+ combinatorial interpretations (see OPAC survey, §11.4), all of them related by relatively easy bijections. But I had my doubts.
Third, I knew quite a bit about efforts to find a combinatorial interpretation for Kronecker coefficients which also generalize LR-coefficients in a different direction. Joint with Greta Panova, I have written extensively on the subject. I was (still am) completely confident that Kronecker coefficients are not in #P for many reasons too long to list (this is Conjecture 9.1 in my OPAC survey). So I simply assumed that Schubert coefficients are also not in #P, by analogy.
Having concluded that one should work in the negative direction, Colleen and I made a major effort towards proving that Schubert coefficients are not in #P, aiming to emulate the strategy in my paper with Chan, and in earlier work with Ikenmeyer and Panova. The basic idea is to show that the positivity problem is not in the polynomial hierarchy PH, which would imply that the counting problem is not in #P. We failed but in a surprisingly powerful way which led me to rethink my whole belief system when it comes to Schubert calculus.
By the Schubert positivity problem we mean the decision problem that Schubert coefficients are positive. This problem is a stepping stone towards finding a combinatorial interpretation, and is also of independent interest. In our previous paper with Colleen, we proved that the positivity problem is in the complexity class AM assuming the Generalized Riemann Hypothesis (GRH). This is a class that is sandwiched between first and second level of the polynomial hierarchy, so in particular in contains NP and BPP, and is contained in Π2. In particular, my dream of proving that the positivity problem is not in PH was doomed from the start (assuming PH does not collapse).
Now that that Schubert positivity is in PH this explains the earlier failures, but leaves many questions. First, should we believe that Schubert coefficients are in #P? That would imply that Schubert positivity is in NP, a result we don’t have. Second, where did we go wrong? Which of my beliefs were mistaken, and what does that say about the rest of my belief system?
Let me start with the second which easier to answer. I continue to stand by our first belief (the miracle of RSK and LR) — this is going nowhere. I am no longer confident in the second belief. It is possible that #P is so much larger than traditional combinatorial interpretations that there is more to the story. And lightnings can strike twice if the buildings are especially tall…
More importantly, I now completely reject the third belief of analogy between Kronecker and Schubert coefficients. While the former is fundamentally representation-theoretic (RT), as our proof shows the latter is fundamentally algebro-geometric (AG). They have nothing in common except for the LR-coefficients. At the end, while we proved that Schubert positivity is in AM (assuming GRH) using the Hilbert’s Nullstellensatz, a key problem in AG.
Faced with a clash of core beliefs, Colleen and I needed to completely rethink the strategy and try to explain what do our results really mean? Turned out, my issues were deeper than I thought. At the time I completely lacked faith in derandomization, which is getting close to be a foundation belief in computational complexity, on par with P ≠ NP. I was even derisive about it in the P.S. to my BBC blog post.
On a personal level, saying that P = BPP is weird, or at least unintuitive. It contradicts everything I know about Monte Carlo methods used across the sciences. It undermines the whole Markov chain Monte Carlo technology I worked on in my PhD thesis and as a postdoc. I even remember a very public shouting match on the subject between the late Steven Rudich and my PhD advisor Persi Diaconis — it wasn’t pretty.
After talking to Avi Wigderson while at IAS, I decided to distance myself and think rationally rather than emotionally. Could P = BPP be really true? Unless you know much about derandomization, even P = ZPP seems unmotivated. But these conjectures have a very good reason in their favor.
Namely, the Impagliazzo–Wigderson’s theorem says that under a reasonable extension of the exponential time hypothesis (EHT), itself an advance extension of P ≠ NP, we have P = BPP. Roughly speaking, if hard NP problems are truly hard (require exponential size circuits), one can simulate binary strings by embedding meshed up solutions into the strings which then look random in a sense of poly-time algorithms can’t tell them apart. This is extremely vague and somewhat misleading — read up more on this in Vadhan’s monograph (Chapter 7).
There is also a CS Theory community based argument. In this 2019 poll conducted by Gasarch, there is near unanimous 98% belief that P = BPP by the “experts” (others people were close to even split). Given that P ≠ NP has 99% belief by the same experts, this crosses from speculation to the standard assumption territory. So it became clear that I should completely switch my core belief from P ≠ BPP to P = BPP.
And why not? I have blindly believed the Riemann Hypothesis (RH) for decades without any in-depth knowledge of analytic number theory beyond a standard course I took in college. I am generally aware of applications of RH across number theory and beyond, see quotes and links here, for example. From what I can tell, RH withstood all attempts to disprove it numerically (going back to Turing), and minor dissents (discussed here) do not look promising.
This all reminded me of a strange dialogue I had with Doron Zeilberger (DZ) over lunch back in October, when we went to celebrate the BBC disproof:
DZ: What conjectures do you believe? Do you believe that RH is true?
IP: I am not sure. Probably, but I don’t have enough intuition either way.
DZ: You are an idiot! It’s 100% true! Do you believe that P ≠ NP?
IP: Yes, I do.
DZ: Ok, you are not a complete idiot.
Anyway, back to the story. I figured that if you believe in RH you may as well believe in GRH. And if you believe in P ≠ NP you may as well believe in ETH. And if you believe in ETH you may as well believe in the Impagliazzo–Wigderson’s Assumption (IWA) which implies that P = BPP. And if you believe in IWA you may as well believe in the Miltersen–Vinodchandran Assumption (MVA) which is an interactive proof version of IWA introduced in this paper, and which implies that NP = AM. Once you break this it into steps, the logic of this implication becomes clear and the conclusion extremely believable.
Having thought through these implications, Colleen and I wrote this note which prompted this blog post. We aim at people in algebraic combinatorics and obtain the following:
Main Theorem [Robichaux–P.] Schubert positivity is in NP (i.e., has a positive rule) assuming GRH and MVA.
The theorem is the closest we got to proving that Schubert coefficients are in #P. The note is written in a somewhat unusual style, explaining the results and refuting potential critiques. Quotes by Poincaré and Voltaire are included in support of the case. Check it out!
In summary, the theorem above completely resolves the Schubert positivity problem albeit conditionally and from computational complexity point of view. It assumes two very hard conjectures, each stronger than a million dollar problem. But so what? It’s not even the first theorem which assumes two million dollars worth of conjectures (it’s a long article — search for “two million dollars”). And with inflation, one million in 2000 is about two millions now, so it’s probably ok to assume two such conjectures in one theorem anyway… 😉
Happy Holidays! Happy New Year! Best wishes everyone!
Concise functions and spanning trees
Is there anything new in Enumerative Combinatorics? Most experts would tell you about some interesting new theorems, beautiful bijections, advanced techniques, connections to other areas, etc. Most outsiders would simply scoff, as in “what can possibly be new about a simple act of counting?” In fact, if you ask traditional combinatorialists they would be happy to tell you they they like their area to be trend-resistant. They wouldn’t use these words, obviously, but rather say something about timeless, or beautiful art, or “balls in boxes”. The following quote is a classic of this genre:
Combinatorialists use recurrence, generating functions, and such transformations as the Vandermonde convolution; others, to my horror, use contour integrals, differential equations, and other resources of mathematical analysis. (J. Riordan, Combinatorial identities, 1968)
If you’ve been reading this blog for a while, then you already know how I feel about such backward-looking views. When these win, the area becomes stale, isolated, and eventually ignored by both junior researchers and the “establishment” (leading math journals, granting agencies, etc.) Personally, I don’t I don’t see this happening in part due to the influence of Theoretical Computer Science (TCS) that I discussed back in 2012 in this blog post.
In fact, the influence of TCS is so great on all aspects of Combinatorics (and Mathematics in general), let me just list three ideas with the most impact on Enumerative Combinatorics:
- Thinking of a “closed formula” for a combinatorial counting function as algorithm for computing the function, leading to Analysis of Algorithms type analysis (see Wilf’s pioneer article and my ICM paper).
- The theory of #P-completeness (and related notions such as #P-hard, #EXP-complete, class GapP, etc.) explaining why various functions do not have closed formulas. This is now a core part of Computational Complexity (see e.g. Chapter 13 in this fun textbook).
- The idea that a “combinatorial interpretation” is simply a function in #P. This is my main direction these days, see this blog post, this length survey and this OPAC talk and this StanleyFest talk.
All three brought remarkable changes in the way the community understands counting problems. In my own case, this led to many interesting question resulting in dozens on papers. Last year, in the middle of a technical complexity theoretic argument, I learned of a yet another very general direction which seem to have been overlooked. I will discuss it briefly in this blog post.
Complete functions
Let A be a set of combinatorial objects with a natural parametrization: A = ∪ An. For example, these can be graphs on n vertices, posets on n elements, regions in the square grid with n squares, etc. Let f: A → N be a function counting objects associated with A. Such functions can be, for example, the number of 3-colorings or the number of perfect matchings of a graph, the number of order ideals or the number of linear extensions of a poset, the number of domino tilings of a region, etc.
We say that f is complete if f(A)=N. Similarly, f is almost complete if f(A) contains all sufficiently large integers. For example, the number of perfect matchings of a simple graph is complete as can be seen from the following nice construction:
Moreover, the number of domino tilings of a region in Z2 is complete since for every integer k, there is a staircase-type region like you see below with exactly k domino tilings (this was observed in 2014 by Philippe Nadeau).
In fact, most natural counting functions are either complete or almost complete. For example, the number of spanning trees of a simple graph is almost complete since the number of spanning trees in an n-cycle is exactly n, for all n>2. Similarly, the number of standard Young tableaux |SYT(λ)| of a partition λ is almost complete since |SYT(m,1)|=m. Many other natural examples are in our paper with Swee Hong Chan (SHC) which started this investigation.
Concise functions
Let f be an almost complete function. We say that f is concise if for all large enough k, there exist an element a ∈ An such that f(a) = k and n < C (log k)c, for some C, c>0. Just like explicit constructions in the context of Graph Theory (made famous by expanders), this notion makes perfect sense irrespectively from our applications in computational complexity (see our paper with SHC linked above).
Note that none of the simple constructions mentioned above imply that the corresponding functions are concise. This is because the size of combinatorial objects is linear in each case, not poly-logarithmic as we need it to be. For the number of perfect matchings, an elegant construction by Brualdi and Newman (1965) shows that one can take n = O(log k). This is the oldest result that we know, that some natural combinatorial counting function is concise.
For the number of domino tilings, SHC and I proved an optimal bound: there is a region with O(log k) squares with exactly k domino tilings. The proof is entirely elementary, accessible to a High School student. The idea is to give explicit transformations k → 2k and k → 2k-1 using gadgets of the following kind:
As always, there are minor technical details in this construction, but the important takeaway is that we obtain an optimal bound, but the regions we construct are not simply-connected. For simply-connected regions the best bound we have is O(log k log log k) for the snake (ribbon) regions, via connection to continued fractions that was recently popularized by Schiffler. Whether one can obtain O(log k) bound in this case is an interesting open problem, see §6.4 in our paper with SHC.
Many concise functions
For the number of spanning trees, whether this function is concise remained an open problem for over 50 years. Even a sublinear bound was open. The problem was recently resolved by Stong in this beautiful paper, where he gave O((log k)3/2/(log log k)) upper bound. Sedláček (1967) conjectured that o(log k) bound for general graphs, a conjecture which remains wide open.
For some functions, it is easy to see that they are not concise. For example, for a partition λ of n, the number of standard Young tableaux |SYT(λ)| is a divisor of n! Thus, for k prime, one cannot take n<k in this case.
Curiously, there exist functions, for which being almost complete and concise are equivalent notions. For example, let T ∈ R2 be a set of n points in general positions in the plane. Denote by g(T) the number of triangulations of T. Is g almost complete? We don’t know but my guess is yes, see Conjecture 6.4 in our paper with SHC. However, we do know exponential lower and upper bounds Cn < g(T) <Dn. Thus, if g is almost complete it is automatically concise with an optimal O(log k) upper bound.
Our final example is much too amusing to be skipped. Let e(P) denote the number of linear extensions of a poset on n elements. This function generalized the number of standard Young tableaux, and appears in a number of applications (see our recent survey with SHC). Tenner proved a O(√k) bound, the first sublinear bound. The conciseness was shown recently by Kravitz and Sah, where they established O(log k log log k) upper bound. The authors conjectured O(log k) bound, but potentially even O((log k)/(log log k)) might hold.
Consider now a restriction of the function e to posets of height two. In our paper with SHC, we have Conjecture 5.17 which claims that such e is still almost complete. In other words, for all large enough k, one can find a poset of height two with exactly k linear extensions. Since the number of linear extensions of such posets is at least (n/2)!2 this would give an optimal bound for general posets as well, so a very sharp extension of the Kravitz–Sah bound. We should mention an observation of Soukup (p. 80), that 13,168,189,439,999 is not the number of linear extensions of a height two poset. This suggests that our conjecture is either false, or likely to be very hard.
Latest news: back to spanning trees
In our most recent paper with Swee Hong Chan and Alex Kontorovich, we resolve one Sedláček’s question and advance another. We study the number τ(G) of spanning trees in a simple planar graph G on n vertices. This function τ is concise by Stong’s theorem (his construction is planar), and it is easy to show by planarity that τ(G) < 6n. Thus, a logarithmic upper bound O(log k) is the best one can hope for. Clearly, proving such result would be a major advancement over Stong’s poly-logarithmic bound.
While we don’t prove O(log k) bound, we do get very close — we prove that this bound holds for the set of integers k of density 1. The proof is both unusual (to me as a combinatorialist), and involves a mixture of graph theory, number theory, ergodic theory, and some pure luck. Notably, the paper used the remarkable Bourgain–Kontorovich technology developed towards the celebrated Zaremba’s Conjecture. You can read it all in the paper or this longish post by Alex.
P.S. Being stubborn and all, I remain opposed to the “unity of mathematics” philosophy (see this blog post which I wrote about the ICM before later events made it obsolete). But I do understand what people mean when they say these words — something like what happened in our paper with Alex and Swee Hong with its interdisciplinary tools and ideas. And yet, to me the paper is squarely in Combinatorics — we just use some funky non-combinatorial tools to get the result.
The power of negative thinking: Combinatorial and geometric inequalities
It’s been awhile since I blogged about mathematics. You know why, of course — there are so many issues in the real world, the imaginary world is just not as relevant as it used to be. Well, at least that’s how I felt until now. But the latest paper we wrote with Swee Hong Chan was so much fun (and took so much effort), the wait is over. There is also some interesting backstory before we can state the result.
What is the inverse problem in Enumerative Combinatorics?
Before focusing on combinatorics, note that inverse problems are everywhere in mathematics. Sometimes they are obvious and stated as such, and sometimes we are so used to these problems we don’t think of them as inverse problems at all. You are probably thinking of major problems (both solved and unsolved), like the inverse Galois problem, Cauchy problem, Minkowski problem or the Alexandrov existence theorem. But really, even prime factorization, integration, taking logs and subtraction can be viewed this way. As I said — they are everywhere.
In Enumerative Combinatorics, a typical problem goes like this: given some set A, find the number N:=|A|. Finding a combinatorial interpretation is an inverse problem: given N, find A such that N=|A|. This might seem silly to an untrained eye: obviously, every nonnegative integer counts something. But it is completely normal to have constraints on the type of solution that you want — this case is no different.
Indeed, if you think about it, the direct problem is not all that well-defined either. For example, do you want an asymptotics or just some kind of bounds on N? Or maybe you want a closed formula? But what is a closed formula? Does it have to be a product formula, or some kind of summation will work? Can it be a multisum with both positive and negative terms? Or maybe you are ok with a closed formula for the generating function in case A=UAn? But what exactly is a closed formula for a GF? The list of questions goes on.
Five years ago, I discussed various different answers to these question in my ICM paper, with ideas goes back to Wilf’s beautiful paper (see also Stanley’s answer). If anything, the answers are not short and sometimes technical. Although my formulations are well-defined, positive results can be hard to prove, while negative results can be really hard to prove. Such is life, I suppose.
So what exactly is a combinatorial interpretation?
It is easy to go philosophical (as Rota does or I do on somewhat broader questions), but let’s focus on math here. I started thinking about the problem when I came to UCLA over twelve years ago, and struggled to find a good answer. I discussed the problem in my Notices paper when I finally made peace with the computational complexity approach. Of the multiple definitions, there is only one that is both convincing, workable and broad enough:
Combinatorial interpretation = #P
I explain the answer in my lengthy OPAC survey on the subject, and in my somewhat entertaining OPAC talk (slides). I have miles to say about this, maybe some other time.
To understand why I case, it’s worth thinking of the origin of the problem. Say, you have an inequality a ≥ b between number of certain combinatorial objects, where a=|A|, b=|B|. If you have a nice explicit injection φ : B → A, this gives a combinatorial interpretation for the defect (a–b) as the number of elements in A without a preimage. If φ and its inverse are computable in polynomial time, this shows that (a–b) counts the number of objects which can be certified to be correct in polynomial time. Thus, the definition of #P.
Now, as always happens in these cases, the reason for the definition is not to give a positive answer (“you know it when you see it” was a guiding principle for a long time), but to give a negative answer. What if many of these combinatorial interpretation problems Stanley discusses in his famous survey simply don’t have a solution? (see my OPAC survey linked above, and this MO discussion for the state of art).
To list my favorite open problem, do Kronecker coefficients g(λ,μ,ν) have a combinatorial interpretation? I don’t believe so, but to give a negative answer we need a definition. There is just no way around it. Note that we already have g(λ,μ,ν)= a(λ,μ,ν) – b(λ,μ,ν) for some numbers of combinatorial objects a and b (formally, these are #P functions). It is the injection that doesn’t seem to work. But why not?
Unfortunately, the universe of “not in #P” results is very small and includes only this FOCS paper with Christian Ikenmeyer and this SODA paper with Christian Ikenmeyer and Greta Panova. Simply put, such results are rare and hard to prove. Let me not explain them, but rather turn in the direction of my current work.
Poset inequalities
Since the inequalities like g(λ,μ,ν) ≥ 0 are so unapproachable in full generality, some four years ago I turned to inequalities on the number of linear extensions of finite posets. Many such inequalities are known in the literature, e.g. the XYZ inequality, the Sidorenko inequality, the Björner–Wachs inequality, etc. It is unclear whether the defect of the XYZ inequality has a combinatorial interpretation, but the other two certainly do (see our “Effective poset inequalities” paper with Swee Hong Chan and Greta Panova).
What we found most interesting and challenging, is the following remarkable Stanley’s inequality on the log-concavity of the number of certain linear extensions:
(this is a slide from my 2021 talk). In a remarkable breakthrough, Stanley resolved the Chung-Fishburn-Graham conjecture using the Alexandrov–Fenchel inequality (more on this later). What I was interesting in the following problem: Is the defect of Stanley’s inequality N(k)^2-N(k-1) N(k+1) in #P? This is still an open problem, and we don’t have tools to resolve it.
It gets worse: in an effort to show that this inequality is in #P, two years ago we introduced a whole new technology of combinatorial atlas. We used this technology to prove a lot new inequalities in this paper with Swee Hong Chan, including multivariate extensions of Stanley inequalities and correlation inequalities. We now know why this technology was never going to apply to the #P problem, but that’s all yet another story.
What we did in our new paper is attacked a similar problem for the generalized Stanley inequality, which has the same statement but with additional constraints that L(xi)=ci for all 1 ≤ i ≤ m, where xi are fixed poset elements and ci are fixed integers. Stanley derived the log-concavity of these more general numbers from the AF inequality in one big swoosh. In our paper, we prove:
Corollary 1.5. The defect of the generalized Stanley inequality is not in #P, for all m ≥ 2 (unless PH collapses to a finite level).
Curiously, in addition to a lot of poset theoretic technology we are using the Yao-Knuth theorem in number theory. Our main result is stronger:
Theorem 1.3. The equality cases of the generalized Stanley inequality are not in PH, for all m ≥ 2 (unless PH collapses to a finite level).
Clearly, if the defect was in #P, then the “defect =? 0″ is in coNP, and the “not in #P” result follows. The complexity theoretic idea of the proof is distilled in our companion paper where we explain why the coincidence problem for domino tilings in R3 is not in PH, and the same holds for many other hard combinatorial problems.
This underscores both the strength and the weakness of our approach. On the one hand, we prove a stronger result than we wanted. On the other hand, for m=0 it is known that the equality cases of the generalized Stanley inequality are in P. This is a remarkable result of Shenfeld and van Handel (actually, a consequence of the their remarkable theory). In fact, we reprove and generalize the result in our combinatorial atlas paper. In the new paper, we prove the m=1 version of this result, using a (also remarkable) followup paper by Ma and Shenfeld. We conjecture that m=2, the defect is already not in #P (Conjecture 10.2), but there seem to be difficult number theoretic obstacles to the proof.
In summary, we now know for sure that the defect of the generalized Stanley inequality does not have a combinatorial interpretation. In particular, there is no direct injective proof similar to that for the Sidorenko inequality, for example (cf. this old blog post). If you are deeply engaged with the subject (and why would you be, obviously?), you are happy. But if not — you probably shrug. Let me now explain why you should still care.
Geometric inequalities
It is rare when when you can honestly say this, but the geometric inequalities really do go back to antiquity (see e.g. here and there), when the isoperimetric inequality in the plane was first discovered. Of the numerous inequalities that followed, note the Brunn–Minkowski inequality and the Minkowski quadratic inequality (MQI) for three convex bodies in R3. These are all consequences of the Alexandrov–Fenchel inequality mentioned above. However, when it comes to equality conditions there is a bit of wrinkle.
For the isoperimetric inequality in the plane, the equality cases are obvious (discs), and there is an interesting history of proofs by symmetrization. For the BM inequality, the equality cases are homothetic convex bodies, but the proof is very far from obvious and requires the mixed volume machinery. For the MQI, the equality conditions were know only in some special cases, and resolved in full generality only recently by Shenfeld and van Handel.
For the AF inequality, the effort to understand the equality conditions goes back to A. D. Alexandrov, who found equality conditions in some cases:
Serious difficulties occur in determining the conditions for equality to hold in the general inequalities just derived. [Alexandrov, 1937]
In 1985, Rolf Schneider formulated a workable conjecture on the equality conditions, which remains out of reach in full generality. He made a strong case for the importance of the problem:
As [AF inequality] represents a classical inequality of fundamental importance and with many applications, the identification of the equality cases is a problem of intrinsic geometric interest. Without its solution, the Brunn–Minkowski theory of mixed volumes remains in an uncompleted state. [Schneider, 1994]
In the remarkable paper mentioned above, Shenfeld and van Handel resolved several special cases of the conjecture. Notably, they gave a complete characterization of the equality conditions for convex polytopes, in a sense of extracting all geometry from the problem, and stating the condition in terms of equality of certain mixed volumes. This is where we come in.
Equality cases of the AF inequality are not in PH
To understand the way Stanley derived his inequality from the AF inequality, it’s worth first explaining the connection to log-concavity:
Stanley considered sections P, Q of the order polytope associated with a given poset and concluded log-concavity for the numbers N(k) via a simple calculation.
Now, our “not in PH” theorem on the equality cases of Stanley’s inequality and this Stanley’s calculation imply that equality cases of the AF inequality are also not in PH (under the same complexity assumptions plus computational setup on how the polytopes are presented). In some sense, this says that the equality cases of the AF inequality can never be fully described, or at least the description by Shenfeld and van Handel is probably the best one can do.
In the spirit of the #P application, our result also implies, that there is unlikely to be a stability result for the AF inequality in full generality (in this sense), see Corollary 1.2 in the paper. Omitting precise statements and technicalities, let us only mention that Bonnesen’s inequality is a basic stability result which can be viewed as a sharp extension of the isoperimetric inequality, including the equality conditions. What we are saying is — don’t expect to ever see anything like that for the AF inequality (see the paper for details).
UPDATE (Feb. 7, 2024). The “m ≥ 6” was later improved to “m ≥ 2“, see our paper on the arXiv. See this video of my Oberwolfach talk on the subject. See also this blog post by Gil Kalai. Note: This paper was accepted to appear at STOC 2024.
UPDATE (Dec 26, 2024). The paper was published in Forum Math. Pi., see here.
Igor Pak's blog 
You must be logged in to post a comment.