Artificial Intelligence Research Data Science Specialist

Hanover, NH

This position works as part of the Dartmouth Libraries Research Data Services team to support research, curricular, and applied artificial intelligence work on campus. The person in this role will bring data science skills together with necessary expertise in information curation and knowledge management to support a variety of generative artificial intelligence applications, such as semantic search, retrieval augmented generation, and information/data retrieval application development. Working alongside campus partners engaged in data science and generative artificial intelligence work, this role will focus on database creation, data ingestion, information preprocessing and embedding, vector database management, and system optimization.

This position is hybrid work location eligible.

Dartmouth College is an equal opportunity/affirmative action employer with a strong commitment to diversity and inclusion. We prohibit discrimination on the basis of race, color, religion, sex, age, national origin, sexual orientation, gender identity or expression, disability, veteran status, marital status, or any other legally protected status. Applications by members of all underrepresented groups are encouraged.

Employment in this position is contingent upon consent to and successful completion of a pre-employment background check, which may include a criminal background check, reference checks, verification of work history, conduct review, and verification of any required academic credentials, licenses, and/or certifications, with results acceptable to Dartmouth College. A criminal conviction will not automatically disqualify an applicant from employment. Background check information will be used in a confidential, non-discriminatory manner consistent with state and federal law.

Dartmouth College has a Tobacco-Free Policy. Smoking and the use of tobacco-based products (including smokeless tobacco) are prohibited in all facilities, grounds, vehicles or other areas owned, operated or occupied by Dartmouth College with no exceptions. For details, please see our policy. https://policies.dartmouth.edu/policy/tobacco-free-policy

Works with researchers, staff, and students to refine the collection and curation of corpus documents to ensure datasets are suitable for artificial intelligence and related computational techniques. Designs database architectures for storing documents and the vector databases that will hold document embeddings. While ensuring database scalability, reliability, and performance optimization, monitors the system's performance and optimizes queries to ensure quick retrieval times and high relevance of retrieved documents. Regularly updates the database with new entries and re-indexes as needed.

Assists researchers, staff and students in the development and application of document preprocessing pipelines to clean and prepare text data for embedding. Automates transcription processing where necessary, including language detection, segmentation, and annotation.

Collaborate with librarians to properly handle metadata and maintain data integrity.

Utilizes machine learning models to generate embeddings from preprocessed text data. Indexes embeddings efficiently within the vector database for fast retrieval. Analyzes retrieval accuracy and optimizes the system by applying query transformations and result reranking techniques.

Provides instruction, outreach, and consultations on advanced computing concepts for faculty, students, and staff to expand computational research skills (including data discovery, curation, management, storage, analysis, visualization, and preservation) as needed for curricular or research projects.

Collaborates with Library Research Data colleagues and Information Technology & Consulting Colleagues to integrate databases effectively with campus AI infrastructure and large language models, and to fine-tune the models based on the data structure and requirements.

Engages in focused professional development activities and serves on applicable Dartmouth committees and task forces, with an emphasis on data science techniques, generative artificial intelligence, and ethical applications of novel technologies. Recommends and facilitates improvements to existing programs and services, and participates in internal training and professional development for Dartmouth Library and related staff.