RESEARCH & DEVELOPMENT

Parameter-Efficient Domain Adaptation via Dual-Adapter Training and Merging: Methods and Evaluation for LLM-Based Financial Analysis Tasks

This thesis addresses the challenge of adapting LLMs to financial analysis through a novel dual-adapter training and merging framework. We focus on Axyon AI’s production system, Alyx, which requires processing diverse financial tasks—stock briefs, news classification, sector analysis, and report generation—each following distinct formatting conventions and reasoning patterns. The key challenge arises from severe sample imbalance in available training data: 1,388 production-critical premarket samples (3.6%) versus 36,782 auxiliary ex-premarket samples (96.4%). Naive combined fine-tuning on this imbalanced distribution causes catastrophic underfitting on minority tasks, degrading performance by 15.2% despite training on 27.5× more data.

A Practical Study of Ensemble Learning for Multi-Horizon Financial Forecasting

This thesis presents a practical study of ensemble learning techniques applied to the field of financial forecasting, conducted at Axyon AI, a fintech company which offers AI-based insights, asset signals and investment strategies.

A central focus is on multi-horizon forecasting: integrating predictions from weak learners trained on different investment horizons in order to improve overall accuracy. The experiments were carried out on the Japan Target Market dataset, across 20-day and 60-day horizons. T

he dataset is based on the Morningstar Japan Target Market Exposure index, which measures the performance of large-cap and mid-cap stocks in Japan, and covers the top 85% of the market by capitalization.

The tested Horizon Union methods demonstrated superior performance when compared to the 20-day single-horizon baseline. These results highlight the practical value of multi-horizon predictions in the context of AI-driven investment strategies.

Listwise Learning to Rank Models with Transformer for Financial Strategies.

Learning to Rank (LTR) is the application of machine learning techniques for ordering items based on relevance, with applications in domains such as search engines, recommendation systems, and natural language processing. Among its various approaches—pointwise, pairwise, and listwise—the listwise framework is particularly suited for capturing interdependencies among ranked items, making it compelling for complex scenarios like asset ranking in investment strategies. This research explores the application of the listwise LTR approach in the financial domain, focusing on ranking assets long-short portfolio construction. In particular, transformers are introduced as the listwise architecture. Transformers have achieved a huge success in the state of the art across various fields due to their self-attention mechanism, which allows them to serve as a context-aware architecture. The implementation includes developing listwise loss functions specifically designed to leverage the model’s ability to incorporate contextual information during both training and inference phases. The study evaluates the performance of the listwise approach compared to traditional pointwise and pairwise methods previously explored by Axyon. A key objective is to assess whether the listwise framework introduces heterogeneity in model predictions, potentially improving the robustness and accuracy of the ensemble rankings. These insights aim to enhance the understanding of inter-asset relationships, ultimately contributing to more effective investment strategies.

A Second-Order Perspective on Model Compositionality and Incremental Learning

The fine-tuning of deep pre-trained models has revealed compositional properties, with multiple specialised modules that can be arbitrarily composed into a single, multi-task model. However, identifying the conditions that promote compositionality remains an open issue, with recent efforts concentrating mainly on linearised networks. We conduct a theoretical study that attempts to demystify compositionality in standard non-linear networks through the second-order Taylor approximation of the loss function. The proposed formulation highlights the importance of staying within the pre-training basin to achieve composable modules. Moreover, it provides the basis for two dual incremental training algorithms: one from the perspective of multiple models trained individually, while the other aims to optimise the composed model as a whole. We probe their application in incremental classification tasks and highlight some valuable skills. In fact, the pool of incrementally learned modules not only supports the creation of an effective multi-task model but also enables unlearning and specialisation in certain tasks.

Enhancing AI Transparency in Investment Management Using Large Language Models

Axyon AI focuses on integrating Generative AI into investment management. The challenge lies in enhancing the transparency and interpretability of AI-driven financial predictions, as existing quantitative methods, such as SHAP, are insufficient for asset managers.

Optimise Heterogeneous Ensemble Search

As Volatility forecasting is critical for risk management and speculative trading, the thesis investigates the application of Machine Learning and Heterogeneous Autoregressive (HAR) models to forecast realised volatility in the highly volatile Brent Crude Oil market. The study evaluates whether ML models outperform traditional HAR models in forecasting Realised Volatility, a volatility estimator based on high-frequency data, by testing both approaches across multiple forecast horizons: one day, one week, and one month.

Optimise Heterogeneous Ensemble Search

Ensemble Learning has become increasingly prevalent as an efficient paradigm in Machine Learning, combining multiple weak models to produce robust predictions across various fields. A key challenge within Ensemble Learning is ensuring diversity among models to explore different data patterns and maintain heterogeneity. This thesis presents a project focused on optimising the parameter search process for heterogeneous ensemble models that incorporate diverse architectures and tasks.

Study and Implementation of Quantum-inspired Boosting Algorithms for AI-powered Financial Asset Management

The development of the Talos web application represents a significant advancement within the corporate context of Axyon AI, providing an intuitive and efficient user interface to streamline the dataset generation process for training ML models.

Study and Implementation of Quantum-inspired Boosting Algorithms for AI-powered Financial Asset Management

This thesis develops and benchmarks a Qboost-based algorithm to enhance Axyon AI’s ensemble learning (EL) pipeline for multi-label classification. EL combines multiple weak learners for more robust predictions, with boosting—an iterative EL method—focusing on training examples where prior models performed poorly, improving stability and accuracy. This project explores adiabatic quantum annealing (AQA) on neutral atom processors, aiming to overcome these limitations.

Unsupervised anomaly detection on time series data

In this project, we aimed at improving a classifier’s performance using the estimated likelihood distribution of a training dataset. The study involved injecting noise into datasets, training classifiers with varying noise levels, estimating data density using unsupervised methods, and exploring relationships between classifier scores, losses, and density estimates. The approach was tested on mixed datasets and financial data, adjusting the classifier's behavior based on density estimates.

Comparison of Learning-To-Rank (LTR) models: Computational Aspects and Application to a Document Ranking Problem

In today's digital landscape, vast accessible resources have prompted the development of efficient Information Retrieval systems using machine learning, specifically through a discipline called Learning to Rank. This approach aims to order information sources for relevant query responses on abundant data, and can be applied beyond recommendation systems and search engines, e.g. to financial investments. This work presents a study on LTR, covering its theory, neural network applications, and practical implementation for document ranking using Python and MSLR-WEB10K dataset.

Out-of-distribution detection methods on deep neural network encodings of images and tabular data

The performance of supervised learning models trained on time-series data can be affected by changing phenomena and drifts. Our prior work in this area focused on detecting outliers in tabular datasets derived from financial time series, yielding promising outcomes but revealing limitations. This project aimed to improve on this by operating in the latent space of deep neural networks, detecting anomalies in the model's internal data representation compared to the learned representations from training data. This shift emphasizes assessing whether the model's data representation is anomalous rather than the input data itself.

Does Catastrophic Forgetting Negatively Affect Financial Predictions?

Nowadays, financial markets produce a large amount of data in the form of historical time series, which quantitative researchers have recently attempted at predicting with deep learning models. These models are constantly updated with new incoming data in an online fashion. However, artificial neural networks tend to exhibit poor adaptability, fitting the last-seen trends, without keeping the information from the previous ones. Continual learning studies this problem, called catastrophic forgetting, to preserve the knowledge acquired in the past and exploiting it for learning new trends. This paper evaluates and highlights continual learning techniques applied to financial historical time series in a context of binary classification (upward or downward trend). The main state-of-the-art algorithms have been evaluated with data derived from a practical scenario, highlighting how the application of continual learning techniques allows for better performance in the financial field against conventional online approaches.

Multivariate Autoregressive Denoising Diffusion Model for Value-at-Risk Evaluation

The Value-at-Risk (VaR) is a common risk measure, often required by financial regulators, typically estimated based on simple closed-form distributions. In this work, we built up on our existing GAN-based model for VaR estimation, by comparing it to newer deep learning approaches, namely an Autoregressive Denoising Diffusion Model based on the Timegrad architecture and a model based on Low-Rank Gaussian Copula Processes.

VaR Estimation with Conditional GANs and GCNs

The problem of Continual Learning has drawn much interest in recent years, as training AI models able to learn new tasks or move to new domains poses the risk of forgetting earlier knowledge. In this study, we have applied several CL methods to train time series forecasting models in the financial domain, using Bayesian changepoint detection methods to segment series into different regimes and thus framing the problem as one of Domain-Incremental Learning. Work presented at Ital-IA 2022.

VaR Estimation with Conditional GANs and GCNs

The Value-at-Risk (VaR) is a common risk measure, often required by financial regulators, typically estimated based on simple closed-form distributions. In this work, we aimed at overcoming the need for parametric assumptions through the use of deep generative models, namely a conditional generative adversarial networks (CGAN). We further extended the model to the multivariate case, by enabling the interaction of multiple stocks through graph convolutions in the generator. Work presented at SIMAI 2021.

FF4 EuroHPC Project Axyon AI - Leveraging HPC for AI and DL-powered Solutions for Asset Management

This is a 15-month research project under the FF4 EuroHPC framework, where Axyon AI leads a consortium of partners including CINECA and AImageLab. The project has the overall goal of improving the service offered by Axyon AI to its clients through several technological advancements. In particular, three main areas of improvement have been identified: computational scalability, risk management and adaptiveness of AI models.

ESAX: Enhancing the Scalability of the Axyon Platform

In this work, carried out jointly with HPC consultants from CINECA, we brought the computational scalability of the Axyon Platform to a new level, almost quadrupling the previous peak of parallely executed jobs. Moreover, we added support to distributed training on multi-GPU/multi-node HPC clusters, and stress-tested our Platform using Marconi100, the 11th largest supercomputer in the world at the time of the project.

Alternative Data for ML-based Asset Performance

Forecasting Alternative data is structured or unstructured data that is not typically used by traditional investment companies and that can provide insights into the future performance of a financial asset. This study examined the possibility of including alternative data sources into Axyon IRIS ML-based predictive models, by comparing the performance before and after the addition of data series extracted from Google Trends.

Reinforcement Learning for Asset Allocation

Reinforcement Learning (RL) has drawn a lot of attention thanks to its successful applications in many fields, most notably to playing games. In this work, we have designed and implemented an RL framework for the task of tactical asset allocation, given a portfolio of equity and fixed income assets. Our approach based on Policy Gradient made use of a particular reward function accounting not only for P&L but also for diversification and stability.

SHAPE Project Axyon AI: a scalable HPC Platform for AI Algorithms in Finance

The goal of this work was to maximize the efficiency of accessing different types of remote computational resources potentially available to our proprietary Machine Learning platform, without losing the flexibility provided by in-house compute power. This is a mandatory requirement for a FinTech company oftentimes working with proprietary data that cannot be uploaded to cloud systems. We achieved this by designing and implementing a scalable and flexible DB-centric Master-Slave system architecture able to exploit any connected internal or external computational resource (including an HPC cluster) in a flawless and secure way. This was the first project that marked a fruitful and ongoing collaboration with CINECA, the largest Italian computing centre.

Extension and Industrialisation of Generative Neural Networks for Financial Time Series Modelling and Forecasting

In this work, we built up on our previous work in generative modelling, extending our GAN model designed for the conditional generation of financial time series. In particular, the contribution of this research activity was twofold: (i) we modified the generator so as to obtain a recurrent sequence-to-sequence architecture, and (ii) we added a self-attention mechanism, bringing improved performance and interpretability.

Deep Generative Neural Networks for Financial Time Series Modelling and Forecasting

In this work, we applied the Generative Adversarial Network (GAN) framework to the challenging task of financial time series generation. We showed how this model can be used to simulate future market scenarios by introducing a conditioning in the generator, using a recurrent neural network. To the best of our knowledge, this was the first application of GANs to financial time series at the time of this work. We presented our results at Nvidia GTC Europe 2018 in Munich.

Deep Learning for Portfolio Allocation

In this work, we combined AI with an existing quantitative portfolio allocation model. In particular, we used the prediction of a Deep Neural Network as “investor views” in the Black-Litterman allocation model. This MSc thesis won the SIAT Technical Analyst Award 2019.

Deep Q-Learning Techniques for Forex Trading

In this work, we applied Reinforcement Learning techniques (and in particular Deep Q-Learning) to the challenging problem of finding profitable trading strategies in the Forex market by trial-and-error in a simulated market. This was Axyon AI’s first of many MSc thesis projects in collaboration with the AImageLab research group at the University of Modena and Reggio Emilia.