Alessandro Achille homepage

Alessandro Achille Applied Scientist at AWS

Caltech

aachille at caltech.edu

alexachi

I am an Applied Scientist working in computer vision and deep learning at Amazon AI (Pasadena) and Caltech (visiting scholar). I graduated in 2019 from the Computer Science Department of UCLA, working with Prof. Stefano Soatto in the Vision Lab. During my PhD I have also been a research scientist intern at Deep Mind and Amazon AI. My research interests include representation learning, information theory, multi-task learning and variational inference.

Before coming to UCLA, I obtained a Master in Pure Math at the Scuola Normale Superiore and the University of Pisa, where I studied model theory, algebraic topology, and their intersection with Prof. Alessandro Berarducci, with particular focus on definable groups in o-minimal theories. During that period, I have also been a visiting student at the University of Leeds Math department.

Teaching

CS103 at Caltech: Topics in Representation Learning, Information Theory and Control

Publications

A. Achille, A. Golatka, A. Ravichandran, M. Polito, S. Soatto

LQF: Linear Quadratic Fine-Tuning

Computer Vision and Pattern Recognition (CVPR), oral, 2021

[ arXiv, bibTeX ]

@misc{achille2020lqf,
      title={LQF: Linear Quadratic Fine-Tuning},
      author={Alessandro Achille and Aditya Golatkar and Avinash Ravichandran and Marzia Polito and Stefano Soatto},
      year={2020},
      eprint={2012.11140},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

A. Golatka, A. Achille, A. Ravichandran, M. Polito, S. Soatto

Mixed-Privacy Forgetting in Deep Networks

Computer Vision and Pattern Recognition (CVPR), 2021

[ arXiv, bibTeX ]

@misc{golatkar2020mixedprivacy,
      title={Mixed-Privacy Forgetting in Deep Networks},
      author={Aditya Golatkar and Alessandro Achille and Avinash Ravichandran and Marzia Polito and Stefano Soatto},
      year={2020},
      eprint={2012.13431},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

G. Paolini, B. Athiwaratkun, J. Krone, J. Ma, A. Achille, R. Anubhai, C. Nogueira dos Santos, B. Xiang, S. Soatto

Structured Prediction as Translation between Augmented Natural Languages

Internetional Conference on Learning Representations (ICLR), spotlight, 2021

[ arXiv, bibTeX ]

@article{paolini2021structured,
  title={Structured Prediction as Translation between Augmented Natural Languages},
  author={Paolini, Giovanni and Athiwaratkun, Ben and Krone, Jason and Ma, Jie and Achille, Alessandro and Anubhai, Rishita and Santos, Cicero Nogueira dos and Xiang, Bing and Soatto, Stefano},
  journal={arXiv preprint arXiv:2101.05779},
  year={2021}
}

H. Harutyunyan, A. Achille, G. Paolini, O. Majumder, A. Ravichandran, R. Bhotika, S. Soatto

Estimating informativeness of samples with Smooth Unique Information

Internetional Conference on Learning Representations (ICLR), 2021

[ arXiv, bibTeX ]

@misc{harutyunyan2021estimating,
      title={Estimating informativeness of samples with Smooth Unique Information},
      author={Hrayr Harutyunyan and Alessandro Achille and Giovanni Paolini and Orchid Majumder and Avinash Ravichandran and Rahul Bhotika and Stefano Soatto},
      year={2021},
      eprint={2101.06640},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

M. Kleinman and A. Achille, D. Idnani, J. C. Kao

Usable Information and Evolution of Optimal Representations During Training

Internetional Conference on Learning Representations (ICLR), 2021

[ arXiv, bibTeX ]

@misc{kleinman2021usable,
      title={Usable Information and Evolution of Optimal Representations During Training},
      author={Michael Kleinman and Alessandro Achille and Daksh Idnani and Jonathan C. Kao},
      year={2021},
      eprint={2010.02459},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

A. Golatkar, A. Achille, S. Soatto

Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations

European Conference on Computer Vision (ECCV), 2020

[ arXiv, bibTeX, code ]

@misc{golatkar2020forgetting,
        title={Forgetting Outside the Box: Scrubbing Deep Networks of Information Accessible from Input-Output Observations},
        author={Aditya Golatkar and Alessandro Achille and Stefano Soatto},
        year={2020},
        eprint={2003.02960},
        archivePrefix={arXiv},
        primaryClass={cs.LG}
    }

Q. Liu, O. Majumder, A. Achille, A. Ravichandran, R. Bhotika, S. Soatto

Incremental Meta-Learning via Indirect Discriminant Alignment

European Conference on Computer Vision (ECCV), 2020

[ arXiv, bibTeX ]

@misc{liu2020incremental,
    title={Incremental Meta-Learning via Indirect Discriminant Alignment},
    author={Qing Liu and Orchid Majumder and Alessandro Achille and Avinash Ravichandran and Rahul Bhotika and Stefano Soatto},
    year={2020},
    eprint={2002.04162},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

A. Golatkar, A. Achille, S. Soatto

Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Neural Networks

Conference on Computer Vision and Pattern Recognition (CVPR), 2020

[ arXiv, bibTeX, code ]

@misc{golatkar2019eternal,
    title={Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networks},
    author={Aditya Golatkar and Alessandro Achille and Stefano Soatto},
    year={2019},
    eprint={1911.04933},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

L. Zancato, A. Achille, A. Ravichandran, R. Bhotika, S. Soatto

Predicting Training Time Without Training

Neural Information Processing Systems (NeurIPS), 2020

[ arXiv, bibTeX, ]

@misc{zancato2020predicting,
      title={Predicting Training Time Without Training},
      author={Luca Zancato and Alessandro Achille and Avinash Ravichandran and Rahul Bhotika and Stefano Soatto},
      year={2020},
      eprint={2008.12478},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}
}

M. Terzi, A. Achille, M. Maggipinto, G. Susto

Adversarial Training Reduces Information and Improves Transferability

AAAI, 2021

[ arXiv, bibTeX, ]

@misc{terzi2020adversarial,
      title={Adversarial Training Reduces Information and Improves Transferability},
      author={Matteo Terzi and Alessandro Achille and Marco Maggipinto and Gian Antonio Susto},
      year={2020},
      eprint={2007.11259},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

K. Gupta, A. Achille, J. Lazarow, L. Davis, V. Mahadevan, A. Shrivastava

Layout Generation and Completion with Self-attention

arXiv preprint

[ arXiv, bibTeX ]

@misc{gupta2020layout,
    title={Layout Generation and Completion with Self-attention},
    author={Kamal Gupta and Alessandro Achille and Justin Lazarow and Larry Davis and Vijay Mahadevan and Abhinav Shrivastava},
    year={2020},
    eprint={2006.14615},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

A. Achille, S. Soatto

Where is the Information in a Deep Neural Network?

arXiv preprint

[ arXiv, bibTeX ]

@ARTICLE{achille2019where,
     author = {{Achille}, Alessandro and {Soatto}, Stefano},
      title = "{Where is the Information in a Deep Neural Network?}",
    journal = {arXiv e-prints},
   keywords = {Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Information Theory, Statistics - Machine Learning},
       year = "2019",
      month = "May",
        eid = {arXiv:1905.12213},
      pages = {arXiv:1905.12213},
archivePrefix = {arXiv},
     eprint = {1905.12213},
primaryClass = {cs.LG},
     adsurl = {https://ui.adsabs.harvard.edu/abs/2019arXiv190512213A},
    adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

A. Golatkar, A. Achille, S. Soatto

Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergence

Neural Information Processing Systems (NeurIPS), 2019

[ arXiv, bibTeX ]

@ARTICLE{golatkar2019time,
       author = {{Golatkar}, Aditya and {Achille}, Alessandro and {Soatto}, Stefano},
        title = "{Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergence}",
      journal = {arXiv e-prints},
     keywords = {Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Statistics - Machine Learning},
         year = "2019",
        month = "May",
          eid = {arXiv:1905.13277},
        pages = {arXiv:1905.13277},
archivePrefix = {arXiv},
       eprint = {1905.13277},
 primaryClass = {cs.LG},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2019arXiv190513277G},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

C. Nguyen, A. Achille, M. Lam, T. Hassner, V. Mahadevan, S. Soatto

Toward understanding catastrophic forgetting in continual learning

NeurIPS '19 Meta-Learning Workshop

[ arXiv, bibTeX ]

@misc{nguyen2019understanding,
    title={Toward Understanding Catastrophic Forgetting in Continual Learning},
    author={Cuong V. Nguyen and Alessandro Achille and Michael Lam and Tal Hassner and Vijay Mahadevan and Stefano Soatto},
    year={2019},
    eprint={1908.01091},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

A. Achille, G. Mbeng ,S. Soatto

Dynamics and Reachability of Learning Tasks

NeurIPS '18 Workshop on Integration of Deep Learning Theories

[ arXiv, bibTeX ]

@ARTICLE{achille2018Dynamics,
       author = {{Achille}, Alessandro and {Mbeng}, Glen and {Soatto}, Stefano},
        title = "{Dynamics and Reachability of Learning Tasks}",
      journal = {arXiv e-prints},
     keywords = {Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Statistics - Machine Learning},
         year = "2018",
        month = "Oct",
          eid = {arXiv:1810.02440},
        pages = {arXiv:1810.02440},
archivePrefix = {arXiv},
       eprint = {1810.02440},
 primaryClass = {cs.LG},
       adsurl = {https://ui.adsabs.harvard.edu/abs/2018arXiv181002440A},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

A. Achille, G. Paolini, G. Mbeng, S. Soatto

The Information Complexity of Learning Tasks, their Structure and their Distance

Information and Inference: A Journal of the IMA, 2020

[ arXiv, bibTeX ]

@ARTICLE{achille2019information,
  author = {{Achille}, Alessandro and {Paolini}, Giovanni and {Mbeng}, Glen and
    {Soatto}, Stefano},
   title = "{The Information Complexity of Learning Tasks, their Structure and their Distance}",
 journal = {arXiv e-prints},
keywords = {Computer Science - Machine Learning, Computer Science - Information Theory, Statistics - Machine Learning},
    year = "2019",
   month = "Apr",
     eid = {arXiv:1904.03292},
   pages = {arXiv:1904.03292},
archivePrefix = {arXiv},
  eprint = {1904.03292},
primaryClass = {cs.LG},
  adsurl = {https://ui.adsabs.harvard.edu/abs/2019arXiv190403292A},
 adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

A. Achille, M. Lam, R. Tewari, A. Ravichandran, S. Maji, C. Fowlkes, S. Soatto, P. Perona

Task2Vec: Task Embedding for Meta-Learning

International Conference on Computer Vision (ICCV), 2019

[ arXiv, bibTeX, code ]

@ARTICLE{achille2019task2vec,
       author = {{Achille}, Alessandro and {Lam}, Michael and {Tewari}, Rahul and
        {Ravichandran}, Avinash and {Maji}, Subhransu and {Fowlkes},
        Charless and {Soatto}, Stefano and {Perona}, Pietro},
        title = "{Task2Vec: Task Embedding for Meta-Learning}",
      journal = {arXiv e-prints},
     keywords = {Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Statistics - Machine Learning},
         year = 2019,
        month = Feb,
          eid = {arXiv:1902.03545},
        pages = {arXiv:1902.03545},
archivePrefix = {arXiv},
       eprint = {1902.03545},
 primaryClass = {cs.LG},
       adsurl = {https://ui.adsabs.harvard.edu/#abs/2019arXiv190203545A},
      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
}

A. Achille, M. Rovere, S. Soatto

Critical Learning Periods in Deep Neural Networks

International Conference on Learning Representations (ICLR), 2019

[ arXiv, bibTeX ]

@inproceedings{
achille2018critical,
title={Critical Learning Periods in Deep Networks},
author={Alessandro Achille and Matteo Rovere and Stefano Soatto},
booktitle={International Conference on Learning Representations},
year={2019},
url={https://openreview.net/forum?id=BkeStsCcKQ},
}

A. Achille, T. Eccles, L. Matthey, C.P. Burgess, N. Watters, A. Lerchner, I. Higgins

Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies

Neural Information Processing Systems (NeurIPS), 2018

[ arXiv, bibTeX ]

@incollection{NIPS2018_8193,
title = {Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies},
author = {Achille, Alessandro and Eccles, Tom and Matthey, Loic and Burgess, Chris and Watters, Nicholas and Lerchner, Alexander and Higgins, Irina},
booktitle = {Advances in Neural Information Processing Systems 31},
editor = {S. Bengio and H. Wallach and H. Larochelle and K. Grauman and N. Cesa-Bianchi and R. Garnett},
pages = {9895--9905},
year = {2018},
publisher = {Curran Associates, Inc.},
url = {http://papers.nips.cc/paper/8193-life-long-disentangled-representation-learning-with-cross-domain-latent-homologies.pdf}
}

A. Achille, S. Soatto

A Separation Principle for Control in the Age of Deep Learning

Annual Reviews of Control, Robotics and Autonomous Systems, 2018

[ arXiv, bibTeX ]

@article{achille2017separation,
    author = { Alessandro  Achille and  Stefano  Soatto},
    title = {A Separation Principle for Control in the Age of Deep Learning},
    journal = {Annual Review of Control, Robotics, and Autonomous Systems},
    volume = {1},
    number = {1},
    pages = {null},
    year = {2018},
    doi = {10.1146/annurev-control-060117-105140},

    URL = {
        https://doi.org/10.1146/annurev-control-060117-105140

    },
    eprint = {
        https://doi.org/10.1146/annurev-control-060117-105140
    }
}

A. Achille, S. Soatto

Emergence of Invariance and Disentangling in Deep Representations

Journal of Machine Learning Research (JMLR), 2018

[ arXiv, bibTeX ]

@article{JMLR:v19:17-646,
  author  = {Alessandro Achille and Stefano Soatto},
  title   = {Emergence of Invariance and Disentanglement in Deep Representations },
  journal = {Journal of Machine Learning Research},
  year    = {2018},
  volume  = {19},
  number  = {50},
  pages   = {1-34},
  url     = {http://jmlr.org/papers/v19/17-646.html}
}

A. Achille, S. Soatto

Information Dropout: learning optimal representations through noisy computation

Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2018

[ arXiv, bibTeX ]

@ARTICLE{achille2018information,
    author={A. Achille and S. Soatto},
    journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
    title={Information Dropout: Learning Optimal Representations Through Noisy Computation},
    year={2018},
    volume={PP},
    number={99},
    pages={1-1},
    keywords={Bayes methods;Information theory;Machine learning;Neural networks;Noise measurement;Training;Representation learning;deep learning;information bottleneck;invariants;minimality;nuisances},
    doi={10.1109/TPAMI.2017.2784440},
    ISSN={0162-8828},
month={},}
}

A. Achille, A. Berarducci

A Vietoris-Smale mapping theorem for the homotopy of hyperdefinable sets

Selecta Mathematica, 2018

[ arXiv, bibTeX ]

@article{achille2018a,
  author = {Achille, Alessandro and Berarducci, Alessandro},
  year = {2018},
  title = {A Vietoris-Smale mapping theorem for the homotopy of hyperdefinable sets},
  journal = {Selecta Mathematica},
  issn = {1022-1824},
  doi = {10.1007/s00029-018-0413-3},
  month = {4},
  pages = {1--29},
  url = {http:https://doi.org/10.1007/s00029-018-0413-3},
  abstract = {Results of Smale (1957) and Dugundji (1969) allow to compare the homotopy groups of two topological spaces X and Y whenever a map f:XâY with strong connectivity conditions on the fibers is given. We can apply similar techniques to compare the homotopy of spaces living in different categories, for instance an abelian variety over an algebraically closed field, and a real torus. More generally, working in o-minimal expansions of fields, we compare the o-minimal homotopy of a definable set X with the homotopy of some of its bounded hyperdefinable quotients X/E. Under suitable assumption, we show that pi_n^def(X)=pi_n(X/E) and dim(X)=dim_R(X/E). As a special case, given a definably compact group, we obtain a new proof of Pillay's group conjecture dim(G)=dim_R(G/G00) largely independent of the group structure of G. We also obtain different proofs of various comparison results between classical and o-minimal homotopy.}
}

Talks

University of Bologna, Bologna, May 2020 Slides (together with Giovanni Paolini)
NeurIPS 2019 Workshop on Information Theory and Machine Learning, Vancouver, December 2019 Slides
Mathematical and Computational Aspects of Machine Learning, Scuola Normale Superiore, Pisa, October 2019 Slides, Notebook (html), Notebook (ipynb) (thanks to Giovanni Paolini)

Earlier work

A. Achille

On definable groups in o-minimal and NIP settings

MSc thesis, University of Pisa, 2015

[ pdf ]