Stroke E.R. DiagnosisA.I. Non-verbal Communication
Research interests: computability of emotions, human-AI interaction, big data, biomedical informatics, visual arts, ...
In the PressStroke E.R. DiagnosisA.I. Non-verbal Communication
Emotion + VisionAn Overview
Emotion Body Lang.A.I. + Computer Vision
Robotics and ArtWorkshop at ICRA
Emotion IntelligenceA.I.
Constable's CloudsApollo Magazine (UK)
Stomata Guard CellsData Science + Biology
Placenta VisionNIH R01 Project
Undergrad. TeachingData Science
Group ActivityField Trip
Doctoral StudentsGroup Meeting
Emotion IntelligenceRobotics
ClusteringData Science Methodology
Doctoral AlumDr. Joshi, IBM Watson
Doctoral AlumDr. Lu, Adobe
Doctoral AlumDr. Joshi, Emmy Award 2023
Doctoral AlumDr. Ye, Amazon Lab126
Emotion IntelligenceData Science
A.I. + ArtsTriangles and Aesthetics
Modeling StormsBig Data
Social ComputingPsychology & Visual Data
van GoghNOVA ScienceNOW
Undergrad. ResearchNSF REU Research Experience
Spot a FakeForbes
ArchaeologyScience Magazine
NSF delegationUS Embassy, Beijing
Inferring AestheticsReported on National TV
The mission of our group is to advance knowledge related to the
analysis, management, and making sense of large and complex visual
data, and to contribute to the society through dissemination of
research findings, developing innovative solutions to real-world
problems that impact society and improve people's lives, and education
of future leaders in this field.
Our research is driven by a strong commitment to multidisciplinary
collaboration, excellence, and ethical responsibility, as we work
towards a future in which machine intelligence enhances and augments
human capabilities. The group has published two monographs and over
70 journal articles
(Google
Scholar profile). Notable technologies developed by the group
include the SIMPLIcity image retrieval
engine (at Stanford), the ALIPR real-time automatic image annotation
system, the Acquine aesthetic quality inference engine, evoked emotion
prediction system, bodily expressed emotion understanding system, and
the brushstroke style analysis methods for oil paintings.
We actively engage Ph.D. and M.S. students in research.
Our past Ph.D. students have achieved great success after graduation, taking on tenure-track faculty and research scientist positions and making significant contributions in fields such as AI, robotics, autonomous driving, computer vision, data science, cloud computing, and computational photography. To learn more about our alumni, please click the "ALUMNI" link below.
During the summers, our graduate students have had the opportunity to intern at leading companies and research institutions such as Adobe Research, Amazon Lab126, Apple, AT&T Labs, Bytedance AI Lab, CMU, DeepMap.AI, Facebook, Google Research, HP Labs, IBM Research, IDIAP Switzerland, Kodak, MathWorks, Merck Research, Nationwide, NEC Research, Pinterest, PNC, Siemens Research, SONY Research, Telefonica R&D Barcelona, Tencent, Toutiao, Twitter, Uber, Verisk Analytics, Xerox PARC, Yahoo!, and 12Sigma.
We welcome talented and motivated students with strong computational and mathematical/statistical skills to apply to join our group.
Our lab actively engages undergraduate students, with a focus on promoting participation from Schreyer Honors Scholars and underrepresented students from the WISER, MURE, FURP, and Millennium Scholars programs. Our undergraduate research students have gone on to pursue graduate studies at esteemed universities including Brown, Carnegie Mellon, Columbia, Cornell, Johns Hopkins, Tufts, Cambridge, and the University of Pennsylvania. Interested students can email Dr. Wang to discuss opportunities. Please be aware of scammers. We do NOT use social media for communication or recruitment. We do NOT use email to advertise any positions. We do NOT ask students to make any payments to us.
Our Logo is a vibrant representation of the multidisciplinary nature of our research foundation. Five colors represent disciplines from which we draw knowledge and methodologies: engineering (orange), science (golden yellow), medicine (green), psychology, art, and humanities (dark blue), and ethics (purple). Light blue symbolizes our educational and training mission. At the intersection of these cores, our work in computer vision ensues. The six light rays, shaped like a camera shutter, hint at our focus on perception and the future.
Since 2000, Dr. Wang has been a faculty member at Penn State, where he holds the position of Distinguished Professor in the Data Science and AI area and (by courtesy) the Human-Computer Interaction area of the College of Information Sciences and Technology. He is also an Affiliated Professor of the Molecular, Cellular, and Integrative Biosciences (MCIBS) Program (Option on Bioinformatics and Genomics, the Huck Institutes of the Life Sciences). He serves as the founding Co-Director of the Intelligent Information Systems Laboratory (2001-). He has served as the lead guest editor of IEEE Trans. on Pattern Analysis and Machine Intelligence Special Section on Real-world Image Annotation and Retrieval (2008), an Associate Editor of IEEE Transactions on Multimedia (2009-2011), an Associate Editor of Computerized Medical Imaging and Graphics (2019- ), a guest editor of IEEE BITS The Information Theory Magazine Special Issue on Information Processing in Arts and Humanities (2022), and an invited speaker at more than 130 institutions. He was a Visiting Professor of Robotics at Carnegie Mellon University (2007-2008) and a Program Manager in the Office of the National Science Foundation Director (2011 and 2012). These other sites [personal; g-scholar; wiki; math genealogy; FAQ] have more information.
Other pictures: 4/2000 Stanford, 9/2007, 4/2010 State College, 9/2010, 12/2013, 12/2014, 12/2015, 12/2017, 12/2019, 9/2020
Sree Bhattacharyya sfb6038 (at) psu.edu Ph.D. Program in Informatics, Penn State, since 2023 Thesis: Committee: to be formed | |
Shuhua Yang sky5341 (at) psu.edu Ph.D. Program in Informatics, Penn State, since 2023 Thesis: Committee: to be formed | |
Manas (Max) Mehta mvm7168 (at) psu.edu Ph.D. Program in Informatics, Penn State, since 2022 Thesis: Committee: James Z. Wang (Advisor), Sharon Huang, Kenneth Huang, Alison Gernand | |
Yimu Pan ymp5078 (at) psu.edu Ph.D. Program in Informatics, Penn State, since 2021 Thesis: Committee: James Z. Wang (Advisor), Sharon Huang, Kenneth Huang, Alison Gernand, Jeffery A Goldstein (Northwestern) | |
Sitao Zhang sitao.zhang (at) psu.edu Ph.D. Program in Informatics, Penn State, since 2020 Thesis: Committee: James Z. Wang (Advisor), Jia Li (Co-Advisor), Sharon Huang, Suhang Wang, Michelle G. Newman | |
Lizhen Zhu ljz5180 (at) psu.edu Ph.D. Program in Informatics, Penn State, since 2020 Thesis: Committee: James Z. Wang (Advisor), Bradley P. Wyble (Co-Advisor), Sharon Huang, Amulya Yadav, Hyungsuk Tak | |
Graduating doctoral advisees are listed under the Alumni tab (above). |
Charantej Reddy Pochimireddy, 2024-
Natalie Victoria Abramov, Computer Science, Schreyer Honors College, 2024-
Radhika Rajesh Nair, Computer Science, Schreyer Honors College, 2024-
Aneesh Singh, Computer Science, Schreyer Honors College, 2022-
Alex H. Piechucki, Data Sciences, Schreyer Honors College, 2022-
Our scientific research has been primarily supported by the US National Science Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Our medical research has been primarily supported by the US National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Among others, we are also supported by Amazon.com, Inc., the National Endowment for the Humanities, and the Houston Methodist Hospital,
Our college and other units at Penn State have been very supportive of our work. Our work uses the ACCESS Cyberinfrastructure (formerly Extreme Science and Engineering Discovery Environment, XSEDE), which is supported by the NSF.
21. Tongan Cai, Machine Learning Scientist, Samsara Inc., San Francisco, CA, to start in late 2024. (g-scholar) cta (at) psu.edu Ph.D. Program in Informatics, Penn State, since 2019, graduation expected in 2024-2025 Thesis: Committee: James Z. Wang (Advisor), Sharon Huang, Justin Silverman, Jia Li | |
20. Dolzodmaa Davaasuren, Software Engineer II (ML), Uber, San Francisco, CA. (g-scholar) dud240 (at) psu.edu, dolzodmaa (at) gmail.com Ph.D. Program in Informatics, Penn State, Fall 2024 (expected) Thesis: (Download) Committee: James Z. Wang (Chair), Sharon Huang, Justin Silverman, Alison Gernand, Charles T. Anderson | |
19. Chenyan Wu, TikTok (Bytedance), CA. (g-scholar) czw390 (at) psu.edu, wcyandzrx (at) gmail.com Ph.D. Program in Informatics, Penn State, Spring 2024 Thesis: Spatial, Temporal, and Morphological Perspectives: Advancing Understanding of Visual Data Depicting Humans (Download) Committee: James Z. Wang (Chair), Sharon Huang, C. Lee Giles, Zihan Zhou (former), Alison Gernand | |
18. Zhuomin Zhang, Applied Scientist II, Amazon, Seattle, WA. (g-scholar) jomine94 (at) gmail.com Ph.D. Program in Informatics, Penn State, Summer 2023 Thesis: Cross-Domain Image Analysis Approaches Towards Segmentation of Placenta Photos and Pictorial Realism Study of Paintings (Download) Committee: James Z. Wang (Advisor), Jia Li (Co-Advisor), Sharon Huang, Suhang Wang, Elizabeth C. Mansfield, Fuqing Zhang (past member) | |
17. Farshid Farhat, Software Engineer, AMD, CA. (g-scholar) farshid.farhat (at) gmail.com Ph.D. Program in Computer Science and Engineering, Penn State, Summer 2021 Thesis: Comprehensive Photographic Composition Assistance through Meaningful Exemplars (Download) Committee: J. Barlow (Advisor), J. Z. Wang (Co-Advisor), J. Li, R. T. Colins, W.-C. Lee, Z. Zhou | |
16. Yu Luo, Research Scientist, Amazon Lab126, Sunnyvale, CA. (g-scholar) yuluo.psu (at) gmail.com Ph.D. Program in Informatics, Penn State, Summer 2021 Thesis: Towards Automated Recognition of Bodily Expression of Emotion in the Wild (Download) Committee: J. Z. Wang (Advisor), J. Li (Co-Advisor), M. B. Rosson, Z. Zhou, R. B. Adams, Jr. | |
15. Mohammad M. Kamani,
Research Scientist II, WYZE Labs, Inc., Kirkland, WA. (g-scholar)
mohamadmahdi.kamani (at) gmail.com Ph.D. Program in Informatics, Penn State, Fall 2020 Academic Computing Fellow, Penn State, 2017-2020 Thesis: Multiobjective Optimization Approaches for Bias Mitigation in Machine Learning (Download) Committee: J. Z. Wang (Advisor), Mehrdad Mahdavi (Co-Advisor), C. L. Giles, X. Zhang, C. E. Forest, J. Li | |
14. Xinye (Rachel) Zheng, Research Scientist, Facebook, Inc., Menlo Park, CA. (g-scholar) zhengxinye (at) gmail.com Ph.D. Program in Informatics, Penn State, Summer 2020 Thesis: Computational Approaches towards Pattern Recognition and Matching for Scientific Images (Download) Committee: J. Z. Wang (Advisor), J. Li (Co-Advisor), S. Abdullah, A. Squicciarini, C. E. Forest | |
13. Yukun Chen, Senior Research Scientist, Instagram (Meta), Menlo Park, CA. (g-scholar) cykustc (at) gmail.com Ph.D. Program in Informatics, Penn State, Spring 2020 Thesis: Aggregated Wasserstein Distance for Hidden Markov Models and Automated Morphological Characterization of Placenta from Photos (Download) Committee: J. Z. Wang (Advisor), J. Li (Co-Advisor), X. Xing, X. Zhang, D. Wang (Northwestern), D. Hunter | |
12. Baris Kandemir, System Software Engineer, NVIDIA, Santa Clara, CA. (g-scholar) bariskand (at) gmail.com Ph.D. Program in Informatics, Penn State, Spring 2019 Thesis: Aesthetic and Affective Applications of Visual Features and Physical Property Analysis of Plant Stomata Through Active Surfaces (Download) Committee: J. Z. Wang (Advisor), J. Li (Co-Advisor), Z. Zhou, R. Adams, Jr., C. Anderson | |
11. Jianbo Ye, Senior Applied Scientist, Amazon AWS AI, Sunnyvale, CA. (g-scholar) jianboye.ai (at) gmail.com Ph.D. Program in Information Sciences and Technology, Penn State, Spring 2018 Thesis: Computational Modeling of Compositional and Relational Data Using Optimal Transport and Probabilistic Models (Download) Committee: J. Z. Wang, J. Li (Co-Chairs), C. L. Giles, Z. Li, R. Adams, Jr. | |
10. Xin Lu, TikTok (Bytedance), CA. (g-scholar) xinlu.psu (at) gmail.com Ph.D. Program in Information Sciences and Technology, Penn State, Spring 2016 Thesis: Visual Characteristics for Computational Prediction of Aesthetics and Evoked Emotions (Download) Committee: J. Z. Wang (Chair), M. B. Rosson, R. Adams, Jr., M. Newman, J. Li | |
9. Yu Zhang, Senior System Software Engineer, NVIDIA, Santa Clara, CA. (g-scholar)
ryanflower (at) gmail.com Ph.D. Program in Information Sciences and Technology, Penn State, Spring 2015 Thesis: Discrete Distribution Clustering in Big Data and a Method for Storm Prediction Leveraging Large Historical Archives (Download) Committee: J. Z. Wang, J. Li (Co-chairs), C. L. Giles, A. Squicciarini, C. E. Forest | |
8. Neela Sawant, Senior Applied Scientist, Alexa Speech, Amazon, Bangalore, India. (g-scholar)
neela.sawant (at) gmail.com Ph.D. Program in Information Sciences and Technology, Penn State, Summer 2013 Thesis: Statistical Modeling of Image Semantics from Imperfectly Labeled Datasets (Download) Committee: J. Z. Wang, J. Li (Co-chairs), C. L. Giles, J. M. Carroll, R. Albert | |
7.
Lei Yao, Software Engineer, Pinterest, San Francisco, CA. (g-scholar)
simplely (at) gmail.com Ph.D. in Information Sciences and Technology, Penn State, Summer 2013 Thesis: Automated Analysis of Composition and Style of Photographs and Paintings (Download) Committee: J. Z. Wang, J. Li (Co-chairs), D. L. Hall, X. (L.) Zhang, P. J. McGrady | |
6.
Mu Qiao, startup, Beijing, China. (g-scholar)
muq103.cse.psu (at) gmail.com Ph.D. in Computer Science and Engineering, Penn State, Fall 2012 Thesis: Mixture Modeling for Complex and Large-Scale Data with Applications (Download) Committee: J. Li (Advisor), J. Z. Wang (Co-advisor), D. Kifer (Co-Chair), J. Barlow, L. Zhang | |
5.
Professor Brian A. Canada, Chair, Department of Computer Science, and Associate Professor with tenure, Computational Science, University of South Carolina, Beaufort, SC.
(g-scholar)
bcanada (at) uscb.edu Ph.D. in Integrative Biosciences, Penn State, Fall 2010 Thesis: Augmentation of the Zebrafish Phenomic Landscape by Histological Analysis and Content-based Image Retrieval (Download) Committee: J. Z. Wang, K. Cheng, K. Weiss (Co-chairs), J. Richtsmeier, W. Miller (Computational Science Minor) | |
4. Ritendra Datta,
Head of Applied AI/Senior Director of Engineering, Databricks, Inc., San Francisco, CA.
(g-scholar)
ritendra.datta (at) gmail.com Ph.D. in Computer Science and Engineering, Penn State, Spring 2009 Thesis: Semantics and Aesthetics Inference for Image Search: Statistical Learning Approaches (Download) Committee: J. Z. Wang (Chair), J. Li (Co-Chair), R. Collins, C. L. Giles, D. Miller, B. Urgaonkar | |
3. Xiaonan Lu, independent scientist, Seattle, WA.
(g-scholar)
xlu (at) cse.psu.edu Ph.D. in Computer Science and Engineering, Penn State, Fall 2008 Thesis: Image Analysis and Metadata Extraction for Document Retrieval (Download) Committee: J. Z. Wang (Chair), L. Giles (Co-Chair), W.-C. Lee, D. Miller, C. Zhu | |
2. Dhiraj
Joshi,
Senior Research Scientist, IBM T. J. Watson Research Center, New York.
(g-scholar)
dhirajjoshi16 (at) gmail.com Ph.D. in Computer Science and Engineering, Penn State, Summer 2007 Thesis: Image Analysis and Retrieval by Spatial Models and Information Integration (Download) Committee: J. Z. Wang (Chair), J. Li (Co-Chair), R. Collins, D. Hunter, W.-C. Lee | |
1. Professor Yixin
Chen,
Chair and Professor, Computer and Information Science, University of Mississippi, Oxford.
(g-scholar)
ychen (at) cs.olemiss.edu Ph.D. in Computer Science and Engineering, Penn State, Summer 2003 Thesis: A Machine Learning Approach to Content-based Image Indexing and Retrieval (Download) Committee: J. Z. Wang (Chair), C. L. Giles, J. Yen, J. Li, D. Richards |
Benjamin Wortman, Data Scientist, Finra bvw5146 (at) psu.edu, Informatics, Penn State, graduated in December 2022 Scholarly Paper: HICEM: A High-Coverage Emotion Model for Artificial Emotional Intelligence | |
Lizhen Zhu, Penn State, PhD candidate ljz5180 (at) psu.edu, Informatics, Penn State, graduated in May 2022 Thesis: Contrastive Visual Learning from Spatiotemporal Information | |
Li Yu, Computer Vision Engineer, Blue River Technology (John Deere) luy133 (at) psu.edu, Informatics, Penn State, graduated in May 2021 Thesis: Explainable Image Classification based on Positive Gradient Distance | |
Siqiong He, Senior Software Engineer, Google sxh415 (at) psu.edu, Information Sciences and Technology, Penn State, graduated in May 2015 Thesis: Modeling Photographic Composition via Triangles | |
Poonam Suryanarayan, Engineering Manager, Foundational Driving, Cruise pzs126 (at) psu.edu, Electrical Engineering, Penn State, graduated in 2011 Thesis: A Compputational Approach for Understanding the Aesthetics of Gray Scale Images | |
AdityaGiri Valluri, Hardware Applications Engineer, Microsemi, CA azv114 (at) psu.edu, Electrical Engineering, Penn State, graduated in May 2009 Thesis: Navigation Through Automated Image Understanding and Retrieval | |
Ashish Parulekar, Director of Data Science and Analytics, Workforce Staffing, Amazon, Arlington, VA Electrical Engineering, Penn State, graduated in December 2005 Thesis: Large-scale Satellite Image Browsing using Automatic Semantic Categorization and Content-based Retrieval | |
Other past members and some current collaborators are listed here.
Our research group concentrates on the interdisciplinary study of A.I., Data Science, Life Sciences, Sustainability, Robotics, Health Care, Mental Health, and Visual Art. We invite you to explore our latest publications in these areas.
This material is presented to ensure the timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.
We provide datasets for research comparison. If you cannot find the data that you need, you may contact Dr. Wang.
Request the BoLD (Body Language Dataset) for modeling emotions
The dataset was created in our bodily expressed emotion understanding work. The data is provided for non-commerical research comparison only. You should not redistribute the data. You are strongly encouraged to participate in our data modeling challenge.
Download
10,000 test images (low resolution web-crawled misc database used in WBIIS);
download
1,000 test images (test database used in SIMPLIcity paper)(zip);
If you need the 60,000-image database used in SIMPLIcity, ALIPR, and ALIP papers for research comparison, please contact James Wang.
The databases should be used only
for comparing algorithms, which can be considered as 'fair use' of
the images and our metadata.
You should not redistribute the images.
These datasets have been used in our publications.
Download the Aesthetics Data for Research (2015)
The dataset was used in our computational aesthetics work. The data is provided for non-commerical research comparison only.
You should not redistribute the data.
Download the Emotion Data for Research (2017)
Three datasets were created and used in our work on modeling evoked emotions. The data is provided for non-commerical research comparison only.
You should not redistribute the data. You will need to contact us to obtain the original images.
Download the Portrait Triangle Data for Research (2017)
The dataset was created and used in our portrait triangle work. The data is provided for non-commerical research comparison only. Copyright of the images belongs to the original photographers. You should not redistribute the data.
Download the Aesthetics Data for Research (2006)
The dataset was used in our computational aesthetics work. The data is provided for non-commerical research comparison only.
You should not redistribute the data.
Download the Brushstroke Data of van Gogh Paintings for Research (tar.gz) and the Manual Brushstroke Extraction data (zip)
The datasets were used in our painting analysis work. The data is provided for academic research comparison only.
You should not redistribute the data.
If you are a government agency, an education institution, or a non-profit organization, we may offer you a FREE license of the SIMPLIcity system to run on Linux or Solaris. Please contact Prof. James Wang by email to discuss details. At the moment, the C source code is not ready for public download. If you are commercial and would like to use SIMPLIcity, let us know and we will try to arrange to let you use.
We are planning to release some other software packages via this site in the future. Stay tuned.
The research group is located in the Westgate Building [about] at Penn State. Please read the FAQ for answers to some common questions.
James Z. Wang, Distinguished Professor