Szymon Woźniak

Models are increasing in size and complexity in the hunt for SOTA. But what if those 2%increase in performance does not make a difference in a production use case? Maybe benefits from a smaller, faster model outweigh those slight performance gains. Also, equally good performance across languages in multilingual tasks is more important than SOTA results on a single one. We present the biggest, unified, multilingual collection of sentiment analysis datasets. We use these to assess 11 models and… Show more

Models are increasing in size and complexity in the hunt for SOTA. But what if those 2%increase in performance does not make a difference in a production use case? Maybe benefits from a smaller, faster model outweigh those slight performance gains. Also, equally good performance across languages in multilingual tasks is more important than SOTA results on a single one. We present the biggest, unified, multilingual collection of sentiment analysis datasets. We use these to assess 11 models and 80 high-quality sentiment datasets (out of 342 raw datasets collected) in 27 languages and included results on the internally annotated datasets. We deeply evaluate multiple setups, including fine-tuning transformer-based models for measuring performance. We compare results in numerous dimensions addressing the imbalance in both languages coverage and dataset sizes. Finally, we present some best practices for working with such a massive collection of datasets and models for a multi-lingual perspective. Show less

Linkage learning is frequently employed in state-of-the-art methods dedicated to discrete optimization domains. Information about linkage identifies a subgroup of genes that are found dependent on each other. If such information is precise and properly used, it may significantly improve a method's effectiveness. The recent research shows that to solve problems with so-called overlapping blocks, it is not enough to use linkage of high quality -- it is also necessary to use many different… Show more

Linkage learning is frequently employed in state-of-the-art methods dedicated to discrete optimization domains. Information about linkage identifies a subgroup of genes that are found dependent on each other. If such information is precise and properly used, it may significantly improve a method's effectiveness. The recent research shows that to solve problems with so-called overlapping blocks, it is not enough to use linkage of high quality -- it is also necessary to use many different linkages that are diverse. Taking into account that the overlapping nature of problem structure is typical for practical problems, it is important to propose methods that are capable of gathering many different linkages (preferably of high quality) to keep them diverse. One of such methods is a Parameter-less Population Pyramid (P3) that was shown highly effective for overlapping problems in binary domains. Since P3 does not apply to permutation optimization problems, we propose a new P3-based method to fill this gap. Our proposition, namely the Parameter-less Population Pyramid for Permutations (P4), is compared with the state-of-the-art methods dedicated to solving permutation optimization problems: Generalized Mallows Estimation of Distribution Algorithm (GM-EDA) and Linkage Tree Gene-pool Optimal Mixing Evolutionary Algorithm (LT-GOMEA) for Permutation Spaces. As a test problem, we use the Permutation Flowshop Scheduling problem (Taillard benchmark). Statistical tests show that P4 significantly outperforms GM-EDA for almost all considered problem instances and is superior compared to LT-GOMEA for large instances of this problem. Show less