Will Linczer | GlyphAI Project

GlyphAI

Data Ingestion & Insight Generation

Description

With the GlyphX MVP approved, the fund board and partners were curious how GlyphX would implement Large Language Model (LLM) AI technology.

My Task

My tasks were to explore LLMs’ data analytics abilities with the GlyphX engineering team, and design an AI feature set that proved useful to potential customers.

Problem Statement

The entire tech industry is scrambling to implement identical LLM chatbots to demonstrate to investors that they are leading the AI charge.

GlyphX needs to reassure its development partners and distinguish itself with a unique LLM feature set that builds models and generates insights faster than ever.

Project

Concept to Design Spec
Feature Design

Team

Project Manager
Product Designer
Engineer Team

Role

Product Designer

Timeline

January - March 2023

Discovery

Activities

LLM Engineering Feasibility Workshop
User Interview Review

Feedback #1: User Frustration

During testing, non-technical users express frustration finding the relevant data to fill templates and build models without assistance.

A research document from the Engineer feasibility workshop with 4 columns. The first describes an LLM ability. The second assigns the relevance of the ability with the colors of red for no, yellow for maybe, and green for yes. The third column explains how the ability is relevant. The fourth describes a hypothetical feature the team could add to GlyphX.

1 of 3 images. A research document from the Engineer feasibility workshop with 4 columns. The first describes an LLM ability. The second assigns the relevance of the ability with the colors of red for no, yellow for maybe, and green for yes. The third column explains how the ability is relevant. The fourth describes a hypothetical feature the team could add to GlyphX.

2 of 3 images. A research document from the Engineer feasibility workshop with 4 columns. The first describes an LLM ability. The second assigns the relevance of the ability with the colors of red for no, yellow for maybe, and green for yes. The third column explains how the ability is relevant. The fourth describes a hypothetical feature the team could add to GlyphX.

Feedback #2: Positive Signs

The engineer's early tests indicate the untrained LLM is on par a with generic junior analyst, able to recognizing macro trends, patterns, and anomalies with ease.

Feedback #3: Misidentified Data

Engineers have noticed the LLM struggling to consistently identify a column’s data type, resulting in misleading analysis.

3 of 3 images. A research document from the Engineer feasibility workshop with 4 columns. The first describes an LLM ability. The second assigns the relevance of the ability with the colors of red for no, yellow for maybe, and green for yes. The third column explains how the ability is relevant. The fourth describes a hypothetical feature the team could add to GlyphX.

Synthesis & Ideation

Activities

High-Fidelity Concept Wireframes

Insight #1: Adjusting Data Ingest

Users needs a tool that automatically informs them which templates are completed when uploading new data.

A layout of the glyphx projects dashboard, with an early mockup of the GlyphAI 'call to action' header

Placing a CTA section of existing features on the dashboard offered a simple starting point, but technical users found it redundant while non-technical users found its guidance lacking.

A layout of the glyphx modeling tool, with an early mockup of the GlyphAI chatbot sidebar opened up on the left

Locating the GlyphAI private chat and its activation button in a persistent location, a distinct sidebar and within the utility bar, gives users a sense of its universal utility.

Insight #2: AI agent Analysis

An LLM-powered agent could assist non-technical users to explore models and generate analyses.

Insight #3: Corrective Training

Users will need tools to provide feedback and correct the LLM for each piece of AI-generated analysis or AI-categorized data

An early mockup of a new data table analysis tool. It is similar to the original glyphx, but the 3D model portion is replaced with a flowchart showing how dat files are connected to each other by GlyphAI. The image also highlights a template pairing tool in the tool sidebar that allows users to provide feedback to GlyphAI

The Data Table editor and the Column Auditor tool, used by technical and non-technical users respectively to correct LLM data categorization, received a positive reception from testers frustrated with LLM errors.

Features

Activities

Developer Specifications

Feature #1: AI Template Updater

The GlyphAI banner informs users of any newly uploaded and unanalyzed data. Once updated, GlyphAI informs the user of any newly available templates.

A product spec version of the GlyphX Template Library, highlighting the finalized 'call to action' header which shows users how many, files and columns the AI has scanned, and how long its been since its been updated

Building non-technical users a 5-click flow from data ingestion to model build provided them with a much needed confidence boost to try out new templates and models on their own.

A product spec version of the GlyphX modeling tool that highlights GlyphAI's integration into the Threads tool

Adding GlyphAI and shareable messages to threads and private chats accentuates the core collaborative ‘multiplayer’ experience without intruding on the GlyphX communication workflow testers expect.

Feature #2: Sharable LLM Chats

Users can access GlyphAI to generate analysis of entire data tables or specific models, and seamlessly share insights or whole conversations with other users.

Feature #3: LLM Feedback Frenzy

Feedback options have been built into each GlyphAI instance, allowing users to review, train, and improve the LLM to better understand their data.

The product spec versions of the GlyphAI sidebar chatbot and the Template detailed view, highlighting the user's ability to provide feedback on every GlyphAI message and decision

While this implementation may seem obvious looking back, a thumbs up/down feedback feature was not in ChatGPT at the time.

Conclusion

Results

The GlyphAI feature presentation, consisting of product specifications and the engineering team's early feasibility testing, convince the venture fund board to approve further funding for GlyphX. More importantly, GlyphX received twelve Letters of Intent from its development partners, achieving it's first paying customers.

Lessons

The GlyphAI design process showcased both the right and wrong ways to collaborate with an engineering team. With the project manager and newly hired CEO focused on the venture board and development parters, I had to maintain the product’s scope.

The engineers at GlyphX are exceptional at pushing boundaries, but introducing a new tool like a Large Language Model can easily derail progress. For instance, while LLMs excel at data cleaning and manipulation, these tasks fall outside GlyphX’s core focus of data exploration, analysis, and communication.

Diluting our early-stage resources to compete with established tools like Excel would have spread us too thin, too fast. By staying focused on our core strengths and avoiding scope creep, I ensured GlyphX remained aligned with its long-term vision without sacrificing momentum in the early stages.