Vincent Tan

News Archive (2022)

Nov 2022: Traveling to NeurIPS 2022. See you in New Orleans!

Nov 2022: Posted a preprint on Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation. Joint work with Ph.D. students Jiawei Du and Yidi Jiang.

Nov 2022: Paper on "Almost Cost-Free Communication in Federated Best Arm Identification" accepted to the 37th AAAI Conference on Artificial Intelligence (AAAI), 2023 (Acceptance Rate: 1721/8777 ≈ 19.6%). Joint work with postdocs Srinivas Kota Reddy and P. N. Karthik. In this paper, we study the problem of best arm identification in a federated learning multi-armed bandit setup with a central server and multiple clients and elucidate the tradeoff between communication cost and number of arm selections.

Nov 2022: Paper on almost optimal variance-constrained best arm identification accepted to the IEEE Transactions on Information Theory. This is joint work with Ph.D. student Yunlong Hou and former student Zixin Zhong. In this paper, we consider the BAI problem under a stringent variance constraint. We propose an algorithm VA-LUCB and show that its performance meets an information-theoretic lower bound.

Nov 2022: Paper on active learning of homogeneous Ising trees accepted to the IEEE Transactions on Information Theory. This is joint work with Ph.D. student Fengzhuo Zhang and former postdoc Anshoo Tandon. In this work, we use active learning techniques to improve the learnability of homogeneous Ising tree models. We show that for high edge correlations, the error exponent can be improved by at least 40% over its passive counterpart.

Nov 2022: Posted a monograph on Common Information, Noise Stability and Their Extensions on the arXiv. Joint work with Lei Yu.

Nov 2022: Technical Program Committee (TPC) Co-Chair of the 2025 International Symposium on Information Theory (ISIT) in Ann Arbor, Michigan.

Oct 2022: Paper on Fast Beam Alignment using Pure-Exploration in Multi-Armed Bandits accepted to the IEEE Transactions on Wireless Communications. Joint work with former CSC scholar Yi Wei and Ph.D. student/postdoc Zixin Zhong. In this paper, we develop a two-phase heteroskedastic track-and-stop algorithm to tackle the beam alignment problem.

Oct 2022: Posted a paper on the effect of pseudo-labeling on the generalization error of the semi-supervised Gibbs algorithm. Joint work with Haiyun He, Gholamali Aminian, Yuheng Bu and Miguel Rodrigues. We show that the generalization performance of SSL with pseudo-labeling is affected not only by the information between the output hypothesis and input training data but also by the information shared between the labeled and pseudo-labeled data samples.

Oct 2022: Posted a paper on Federated Best Arm Identification with Heterogeneous Clients. Joint work with PhD. student Zhirui Chen and postdoc P. N. Karthik. In this paper, we show, among other results, that for any algorithm whose upper bound on the expected time to find the best arms matches with the lower bound up to a multiplicative constant, the ratio of any two consecutive communication time instants must be bounded.

Oct 2022: Selected as a "Top Reviewer" for NeurIPS 2022.

Oct 2022: Posted a paper on Understanding and Mitigating Dimensional Collapse in Heterogeneous Federated Learning. Joint work with Ph.D. student Yujun Shi and other co-authors. In this paper, we address the data heterogeneity problem in federated learning by mitigate dimensional collapse.

Sep 2022: Received the NUS Annual Teaching Excellence Award (ATEA).
Sep 2022: Posted a paper on Relational Reasoning via Set Transformers with applications to multi-agent RL (MARL) which has been accepted to NeurIPS 2022. Joint work with Ph.D. student Fengzhuo Zhang and co-authors. In this paper, we present theoretical results justifying the use of transformers in cooperative MARL. We show that, with this choice of architecture, sub-optimality gaps grow independently of the number of agents.

Sep 2022: Three papers accepted to NeurIPS 2022! One paper with Junwen Yang on Best Arm Identification in Linear Bandits, one paper with Du Jiawei (and co-authors) on Sharpness-Aware Training, and the last with Fengzhuo Zhang (and co-authors) on Relational Reasoning via Set Transformers with applications to MARL. See CS Conference Papers for details.

Sep 2022: Paper on Exact Recovery in the General Hypergraph Stochastic Block Model accepted to the IEEE Transactions on Information Theory. This is joint work with former postdoc Qiaosheng Zhang. In this paper, we characterize the fundamental limits for recovering k hidden communities based on an observed hypergraph. The phase transition is represented in terms of a quantity which we term as the generalized Chernoff--Hellinger divergence between communities. Our algorithm consists of a spectral clustering step and successive local refinement steps.

Aug 2022: Paper on Information-Theoretic Characterization of the Generalization Error for Iterative Semi-Supervised Learning accepted to the Journal of Machine Learning Research! This is joint work with Ph.D. students Haiyun He and Hanshu Yan. In this paper, we characterize the generalization error of iterative semi-supervised learning (SSL) algorithms that iteratively generate pseudo-labels for the unlabelled samples. Our theoretical results suggest that when the class conditional variances are not too large, the gen-error decreases with the number of iterations t, but quickly saturates. On the other hand, when the class conditional variances are large, the gen-error increases with t.

Aug 2022: Lectures for EE2012A Analytical Methods for ECE posted on Youtube. Thanks to EE2012A student Valencia Chong for helping with this.

Aug 2022: Posted a preprint "Almost Cost-Free Communication in Federated Best Arm Identification". Joint work with Srinivas Kota Reddy and P. N. Karthik. In this paper, we study the problem of best arm identification in a federated learning multi-armed bandit setup with a central server and multiple clients and elucidate the tradeoff between communication cost and number of arm selections.

Aug 2022: Congrats to my student Hanshu Yan for the successful defense of his Ph.D. thesis titled "Towards Adversarial Robustness for Deep Vision Systems".

Aug 2022: Congrats to my student Haiyun He for the successful defense of her Ph.D. thesis titled "Fundamental Performance Limits of Statistical Problems: From Detection Theory to Semi-Supervised Learning".

Aug 2022: Three papers accepted to the IEEE Information Theory Workshop (ITW) in Mumbai, India. See Conference Papers for details.

Jul 2022: Invited to be an Area Chair of the 2023 International Conference on Learning Representations (ICLR)

Jul 2022: Gave a tutorial with Zixin Zhong on pure exploration in multi-armed bandits at IJCAI-ECAI 2022. [Slides]

Jul 2022: Gave a talk on variance-constrained best arm identification at the S3 Optimization Day at IORA. [Slides] [Video (My talk starts at 1:43)]

Jun 2022: Posted a preprint on establishing the asymptotic Nash equilibrium for the M-ary sequential adversarial hypothesis testing game. Joint work with PhD student Jiachun Pan and postdoc Yonglong Li.

May 2022: Rated 4.9/5.0 for EE5137 Stochastic Processes and 4.5/5.0 for EE2012A Analytical Methods in ECE in Spring 2022. See Teaching.

May 2022: Posted a paper on sharpness-aware training for free. Joint work with Ph.D. student Jiawei Du and co-authors.

May 2022: Joined the editorial board of reviewers of the Journal of Machine Learning Research.

May 2022: Delivered a plenary talk at the 2022 National Conference on Communications (NCC), India.

May 2022: Delivered an invited talk on Bandit Online Clustering at the Stanford RL seminar series [Video].

May 2022: Delivered a tutorial with Cédric Févotte on "Recent Advances in Nonnegative Matrix Factorization" at ICASSP 2022 [Slides].

May 2022: Paper MC2G: An Efficient Algorithm for Matrix Completion with Social and Item Similarity Graphs accepted to the IEEE Transactions on Signal Processing. Joint work with former postdoc Qiaosheng Zhang and KAIST collaborators Geewon Suh and Changho Suh.

Apr 2022: Honoured to receive the university level Annual Teaching Excellence Award (ATEA) for 2022.

Apr 2022: Three papers accepted to ISIT 2022. See Conference Papers for details.

Apr 2022: Paper on adversarial robust deep image denoising accepted to IJCAI 2022 as a short oral presentation. This work systematically investigates the adversarial robustness of deep image denoisers. Work done by Hanshu Yan and co-authors.

Apr 2022: Survey paper on risk-aware multi-armed bandits accepted to the survey track of IJCAI 2022. Joint work with Prashanth L. A. and Krishna Jagannathan.

Apr 2022: Monograph titled "Common Information, Noise Stability, and Their Extensions" accepted to the Foundations and Trends® in Communications and Information Theory. Joint work with Lei Yu. In this monograph, we review Wyner's and Gács-Körner-Witsenhausen's common information. We then discuss their extensions to the Rényi and exact common information as well as the nonnegative rank. Finally, we discuss the noise stability and non-interactive correlation distillation problems and connect them to contemporary conjectures in information theory and discrete probability, such as the Courtade-Kumar, Li-Médard and Mossell-O’Donnell conjectures.
Mar 2022: Posted a paper on best arm identification in restless Markov multi-armed bandits. Joint work with postdocs P. N. Karthik and Srinivas Kota Reddy. In this work, we provide problem instance-dependent asymptotic bounds on the growth rate of the expected time required to find the index of the best arm where the arms evolve according to Markov chains.

Mar 2022: Paper on sequential composite hypothesis testing under probabilistic constraints on the stopping time accepted to the IEEE Transactions on Information Theory. Joint work with PhD student Jiachun Pan and postdoc Yonglong Li. In the second-order results herein, one of our main technical contributions is the derivation of a central limit-type result for a maximum of an uncountable set of log-likelihood ratios.

Mar 2022: Reappointed as a Dean's Chair Associate Professor in the College of Design and Engineering.

Mar 2022: Appointed as a Senior Area Editor of the IEEE Transactions on Signal Processing.

Mar 2022: Paper on class incremental learning (CIL) accepted to the 2022 Conference on Computer Vision and Pattern Recognition (CVPR) (acceptance rate: 2067/8161 ≈ 25.33%). Joint work with PhD student Yujun Shi and collaborators. In this paper, we propose a Class-wise Decorrelation regularizer that enables CIL learner at the initial phase to mimic representations produced by the oracle model (the model jointly trained on all classes) and thus boosting the performance of CIL.

Feb 2022: Paper with Yonglong Li and Marco Tomamichel on optimal adaptive strategies for sequential quantum hypothesis testing accepted to the Communications in Mathematical Physics. In this paper, we study adaptive and non-adaptive strategies for sequential quantum hypothesis testing and provide evidence showing that adaptive measurements outperform non-adaptive ones.

Feb 2022: Paper with Joel Chang on unification of Thompson sampling algorithms for risk-aware bandits selected for an oral presentation at AAAI 2022.

Feb 2022: Selected as a "top reviewer" for AISTATS 2022.

Feb 2022: Posted a paper on online clustering with bandit feedback. Joint work with Junwen Yang and Zixin Zhong. In this paper, we propose a computationally efficient and asymptotically optimal algorithm for online clustering with bandit feedback.

Feb 2022: Awarded an MOE AcRF Tier 2 grant titled "Learning Latent Structure of High-Dimensional Data with Adversarial Training" (476k over three years)!

Feb 2022: Awarded an MOE AcRF Tier 1 grant titled "Information-Theoretic Limits for Online Learning and Adversarial Optimization" (210k over three years)!

Jan 2022: Heartiest congratulations to Qiuyu Zhu for a successful defense of his PhD thesis titled "Online Resource Allocation and Its Applications"!

Jan 2022: Posted a paper on variance-constrained best arm identification. Joint work with Yunlong Hou and Zixin Zhong. In this paper, we consider the BAI problem under stringent variance constraints. We propose an almost asymptotically optimal algorithm VA-LUCB.

Jan 2022: Honored to receive the Engineering Educator Award for AY2020/21.

Jan 2022: Will deliver a plenary talk at the 2022 National Conference on Communications (NCC), India.

Jan 2022: Paper on efficient sharpness-aware minimization accepted to the 10th International Conference on Learning Representations (ICLR) (acceptance rate: 1095/3391 ≈ 32.3%). Joint work with Ph.D. students Jiawei Du and Hanshu Yan and colleagues from A*STAR. In this paper, we use several techniques, including Stochastic Weight Perturbation and Sharpness-Sensitive Data Selection to reduce the computational burden of sharpness-aware minimization or SAM.

Jan 2022: Posted a paper on adversarial robust deep image denoising. Work done by Hanshu Yan and co-authors.

Jan 2022: A paper on online noisy maximization of a Brownian motion accepted by the IEEE Transactions on Signal Processing. Joint work with Zexin Wang and Jonathan Scarlett. This paper considers the regret in the Bayesian optimization of a one-dimensional Brownian motion in which the T adaptively chosen observations are corrupted by Gaussian noise.

News Archive (2021)

Dec 2021: Rated 4.6 (out of 5.0) for lectures and tutorials for teaching MA4270 in Fall 2021. See review quotes here.

Dec 2021: Posted a paper on class incremental learning (CIL). Joint work with PhD student Yujun Shi and collaborators. In this paper, we propose a Class-wise Decorrelation regularizer that enables CIL learner at the initial phase to mimic representations produced by the oracle model (the model jointly trained on all classes) and thus boosting the performance of CIL.

Dec 2021: Paper on unification of Thompson sampling algorithms for risk-aware bandits accepted to AAAI 2022 (15% acceptance rate). Joint work with undergraduate student Joel Chang. In this paper, we design and analyze asymptotically optimal TS-based algorithms for regret minimization in risk-aware bandits. The family of risk measures considered is fairly general, and includes various common ones like the CVaR and the proportional hazard.

Nov 2021: A paper on smooth conditional Rényi entropies accepted by the IEEE Transactions on Information Theory. Joint work with Yuta Sakai. In this paper, we introduce a new quantity called the conditional smooth-* entropy (the * stands for STAR or Sakai–Tan–Arimoto–Rényi). This quantity, as well as related notions, turns out to admit operational meanings in terms of the Campbell's source coding problem, the Arıkan–Massey guessing problem, and the Bunte–Lapidoth task encoding problem.

Nov 2021: Will be delivering a tutorial on recent advances in nonnegative matrix factorization with Cédric Févotte at ICASSP 2022.

Nov 2021: Posted a paper on covert communication with mismatched decoding. Joint work with Qiaosheng Zhang. In this paper, we derive bounds on the covert capacity when the decoder is fixed. We deduce, among other things, the covert capacity under the erasures-only decoding metric.

Oct 2021: Posted a paper on active learning to boost the error exponent of learning homogeneous Ising trees. Joint work with Fengzhuo Zhang and Anshoo Tandon. In this work, we use active learning techniques to improve the learnability of homogeneous Ising tree models. We show that for high edge correlations, the error exponent can be improved by at least 40% over its passive counterpart.

Oct 2021: Posted a paper on the Pareto frontier of regret minimization and best arm identification in stochastic bandits. Joint work with Zixin Zhong and Wang Chi Cheung. In this work, we establish the optimal tradeoff (Pareto frontier) between best arm identification and regret minimization. We also propose an algorithm BOBW-lil-UCB that almost achieves the optimal tradeoff.

Oct 2021: Posted a paper on efficient sharpness-aware minimization. Joint work with Ph.D. students Jiawei Du and Hanshu Yan and colleagues from A*STAR. In this paper, we use several techniques, including Stochastic Weight Perturbation and Sharpness-Sensitive Data Selection to reduce the computational burden of sharpness-aware minimization or SAM.

Oct 2021: Posted a paper on information-theoretic generalization bounds for iterative semi-supervised learning. Joint work with Ph.D. students Haiyun He and Hanshu Yan. In this work, we show, theoretically and empirically, that when the class conditional variances are not too large, the upper bound on the generalization error decreases monotonically with the number of iterations, then quickly saturates.

Sep 2021: Will deliver a plenary talk at next year's National Conference on Communications (NCC), India.

Sep 2021: Paper on third-order asymptotics of variable-length compression allowing errors accepted by the IEEE Transactions on Information Theory. Joint work with Yuta Sakai (University of Hyogo) and Recep Can Yavas (Caltech). In this paper, we show that the third-order term in the asymptotic expansion for non-prefix-free variable-length compression with errors is -((1-ε)log n)/2, where ε and n are the allowable error probability and blocklength respectively.

Sep 2021: Paper on robustifying latent tree learning algorithms with vector variables accepted to NeurIPS 2021. Congrats to Ph.D. student Fengzhuo Zhang on his first paper at NUS. In this paper, we analyze the performance of classical latent tree learning algorithms (such as (Spectral) Neighbor Joining and Chow-Liu Recursive Grouping) when the observations are adversarially corrupted. A novel converse is also developed, showing that the performance bounds are tight in certain asymptotic regimes.

Sep 2021: Heartiest congratulations to Zixin Zhong for the successful defense of her Ph.D. thesis titled "Performance Guarantees for Online Learning: Cascading Bandits and Adversarial Corruptions". Co-advised with Wang Chi Cheung.

Sep 2021: Paper on Thompson sampling for cascading bandits accepted by the Journal of Machine Learning Research. Joint work with Zixin Zhong and Wang Chi Cheung. We develop Thompson sampling-based algorithms and minimax lower bounds for cascading bandits and its linear generalization.

Sep 2021: Submitted a monograph on Common Information, Noise Stability, and Functional Inequalities to the Foundations and Trends on Communications and Information Theory. Joint work with Lei Yu. Email me for a copy if you're interested in it.

Sep 2021: The group will be presenting three works at the upcoming Beyond IID conference.

Aug 2021: Posted a new paper on a unification of Thompson sampling algorithms for risk-aware bandits. Joint work with Joel Chang.

Aug 2021: Congrats to PhD student Haiyun He for winning the poster prize at the 2021 East Asian School of Information Theory

Aug 2021: Two papers, one on quantum sequential hypothesis testing and another on distributed sequential hypothesis testing with zero rate compression, accepted to the 2021 IEEE Information Theory Workshop. See conference papers.

Aug 2021: Presented a tutorial on common information (Wyner's, Gács--Körner--Witsenhausen's, Noninteractive Correlation Distillation) at the 2021 East Asian School of Information Theory [Slides].

Jul 2021: Tutorial titled “Common Information: Old and New” at the 2021 International Symposium On Inform. Theory (ISIT) with Lei Yu (Nankai University) [Slides].

Jul 2021: Conference paper An Interpretable Intensive Care Unit Mortality Risk Calculator accepted by the 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC 2021). Joint work with Eugene Ang (Math HYP student), Yong Sheng Soh and Milashini Nambiar.

Jul 2021: Paper on state masking over a compound channel accepted by the IEEE Transactions on Information Theory. Joint work with friends from Iran Sadaf Salehkalaibar, Mohammad Hossein Yassaee and Mehrasa Ahmadipour. This paper derives fundamental limits of reliable communication over a two-state compound channel when the state of the channel needs to be masked. We use techniques from covert communications.

Jul 2021: Videos for EE5137 Stochastic Processes taught in the Spring of 2021 uploaded to Youtube.

Jun 2021: Reflections on Practice Paper Meeting the Bar in Teaching for Tenure-track Assistant Professors published in the Asian Journal of the Scholarship of Teaching and Learning. In this reflections paper, I share my thoughts on how research-active faculty can meet the basic requirements of being an effective educator.

Jun 2021: Major grant awarded! DESCARTES: A CREATE Program on AI-based Decision making in Critical Urban Systems
Workpackage 3: Optimization-Driven Hybrid AI
Lead PI (with Caroline Chaux-Moulin), 10/2021 — 09/2026, ~S$3.6 mil (out of ~€35 mil)

Jun 2021: Paper on change point detection with training sequences accepted by the IEEE Transactions on Information Theory. Joint work with Haiyun He and Qiaosheng Zhang. This paper discusses the classical offline change point detection problem but with the caveat that the pre- and post-change distributions are only known through available training samples.

Jun 2021: Paper on adversarially trained NMF accepted by the IEEE Signal Processing Letters. Joint work with Research Assistant Ting Cai and Cédric Févotte. Congrats to Ting Cai on her first paper. This paper discusses an adversarially-trained version of NMF. On benchmark datasets, the prediction results are superior to existing NMF algorithms.

Jun 2021: Paper on algorithms for positive semidefinite factorization published in the IEEE Transactions on Signal Processing. Joint work with Dana Lahat, former HYP student Yanbin Lang and Cédric Févotte. This paper connects the positive semidefinite matrix factorization problem with some signal processing primitives including affine rank minimization and phase retrieval.

Jun 2021: Congrats to UROP student Joel Chang for winning the Outstanding Undergraduate Researcher Prize. He worked on CVaR risk-constrained bandits with Qiuyu Zhu and myself and concurrently improved our ICML 2020 result.

Jun 2021: Paper on covert communication and identification via channels (two of my favorite topics in Shannon theory) accepted to the IEEE Transactions on Information Theory. Joint work with Qiaosheng Zhang.

May 2021: Paper on Analysis of Optimization Algorithms via Sum-of-Squares (SOS) accepted to the Journal of Optimization Theory and Applications. Joint work with former M.Eng. student Sandra Tan and former postdoc Antonios Varvitsitis.

May 2021: Three papers accepted to the 2021 International Conference on Machine Learning (ICML). See CS Conference Papers.

Apr 2021: Two papers accepted to the 2021 International Symposium on Information Theory (ISIT). See Conference Papers.