← Atlassian Interview InsightsMachine Learning Engineer

Atlassian·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say
Jun 2026

Summary

Atlassian ML Engineer system design round focused entirely on building an intelligent ticket routing service, basically a Jira-flavored ML platform question. Pretty intense scope for a single session and I definitely underestimated how deep they wanted to go on the ML side.

Questions Asked (4)

Q1

Design a service that lets client companies submit bug tickets and automatically classifies and routes each ticket to the correct internal team. Walk through functional requirements, architecture, the ML model, and how you'd handle scaling and mis-routes.

System DesignTechnical Trade-offs
Author's notes

This is a big one.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying functional and non-functional requirements, then propose a high-level architecture that separates ingestion, classification, and routing. Dive into the ML model design, including feature engineering, model selection, and evaluation metrics, and finally discuss scaling strategies and how to handle mis-routes with feedback loops.

Pro tip: Emphasize the importance of a human-in-the-loop system for continuous improvement and the need to balance automation with accuracy, especially in a multi-tenant environment like Atlassian's.

1. Clarify Requirements

Ask questions to understand the scale, latency, accuracy needs, and integration points with existing systems like Jira. Define functional requirements (e.g., ticket submission, classification, routing) and non-functional requirements (e.g., throughput, latency, multi-tenancy).

2. High-Level Architecture

Outline components: API gateway for ticket submission, message queue for decoupling, preprocessing service, ML model service, routing service, and feedback storage. Consider using a microservices architecture for scalability and maintainability.

3. ML Model Design

Discuss model choice (e.g., fine-tuned BERT for text classification), feature engineering (text, metadata), training data (historical tickets), and evaluation metrics (accuracy, F1, confusion matrix). Address class imbalance and multi-label scenarios.

4. Scaling and Reliability

Explain how to scale horizontally using Kubernetes, auto-scaling, and caching. Discuss model serving with TensorFlow Serving or similar, and ensure low latency with batch processing or async queues. Address multi-tenancy and data isolation.

5. Handling Mis-routes and Feedback

Propose a feedback loop where mis-routed tickets are corrected by humans and used for retraining. Implement confidence thresholds to route uncertain tickets to a human triage queue. Monitor model performance and drift.

Key Points to Mention

  • Multi-tenancy and data isolation for client companies
  • Choice of ML model (e.g., BERT) and handling of text and metadata features
  • Scalability via microservices, message queues, and auto-scaling
  • Feedback loop for continuous model improvement and handling mis-routes
  • Integration with existing ticketing systems (e.g., Jira) and APIs
  • Evaluation metrics and monitoring for model performance and drift

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

What features would you use to train the classification model, and how would you handle cases where the model's confidence is low?

Technical Trade-offsProduct Analytics & Metrics
Author's notes

I went with title, description, component field, and customer tier as features which felt reasonable.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the classification task and available data, then propose a feature set that balances predictive power with engineering feasibility, covering text, behavioral, and metadata signals. For low-confidence cases, describe a tiered strategy: threshold-based fallback, human-in-the-loop review, and active learning to improve the model over time.

Pro tip: Frame low confidence as a product opportunity, not a failure—tie it to Atlassian's values like 'Don't #@!% the customer' by suggesting graceful degradation and transparent UX. Also, mention monitoring confidence distribution as a proxy for data drift.

1. Clarify the problem and data

Ask about the classification task, label availability, and data sources (e.g., Jira tickets, Confluence pages, user actions). This shows you avoid jumping to solutions without context.

2. Propose feature categories

Outline features across text (TF-IDF, embeddings), behavioral (clickstream, edit history), metadata (priority, component), and graph-based (collaboration networks). Prioritize based on signal and cost.

3. Address low-confidence handling

Describe a multi-tiered approach: set a confidence threshold, route low-confidence predictions to a human review queue or fallback rule, and log them for active learning.

4. Tie to product metrics and iteration

Explain how you'd measure success (e.g., precision/recall, review rate, user satisfaction) and iterate by retraining with reviewed examples to reduce low-confidence cases over time.

Key Points to Mention

  • Feature engineering trade-offs: text embeddings vs. bag-of-words, real-time vs. batch features, and cost of computation.
  • Confidence calibration: using Platt scaling or isotonic regression to make confidence scores reliable.
  • Threshold selection: based on business cost of false positives vs. false negatives, not just default 0.5.
  • Human-in-the-loop: designing a review workflow that captures labels for active learning.
  • Fallback strategies: default actions, rule-based overrides, or asking the user for clarification.
  • Monitoring: tracking confidence distribution and low-confidence rate as a signal for model drift or data quality issues.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

How would you track SLA compliance and surface status updates back to the submitting company?

System DesignAPI & Integrations
Author's notes

Shorter exchange on this one.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining what SLAs mean in the context of ML services (e.g., latency, availability, accuracy) and how they are measured. Then describe a system that continuously monitors these metrics, evaluates compliance against thresholds, and proactively pushes status updates to the submitting company via their preferred channels. Emphasize observability, alerting, and integration with existing Atlassian tools.

Pro tip: Tie your answer to Atlassian's ecosystem—mention how you'd leverage tools like Jira, Opsgenie, or Statuspage for incident management and status communication, showing you understand their product suite and can build cohesive solutions.

1. Define SLAs and Metrics

Clarify the specific SLAs (e.g., uptime, p95 latency, model accuracy) and how they are measured, including data sources and aggregation windows.

2. Implement Monitoring and Alerting

Set up real-time monitoring pipelines to collect metrics, evaluate compliance, and trigger alerts when thresholds are breached or predicted to breach.

3. Design Status Update Mechanism

Create a system to surface status updates (e.g., dashboards, webhooks, email) that automatically notifies the submitting company of SLA status and incidents.

4. Integrate with Atlassian Tools

Leverage Atlassian products like Jira for issue tracking, Opsgenie for alerting, and Statuspage for public status communication to streamline workflows.

5. Ensure Feedback and Iteration

Collect feedback from the submitting company, review SLA breaches, and continuously improve the monitoring and communication processes.

Key Points to Mention

  • SLI/SLO/SLA hierarchy and error budgets
  • Real-time monitoring and observability tools (e.g., Prometheus, Grafana)
  • Automated alerting and incident management (e.g., PagerDuty, Opsgenie)
  • API/webhook integrations for status updates
  • Atlassian ecosystem integration (Jira, Statuspage, Opsgenie)
  • Feedback loops and continuous improvement

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q4

How would you monitor the system in production and detect when the classification model starts degrading?

Product Analytics & MetricsRoot Cause Analysis
Author's notes

Blanked for a second here.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer around a layered monitoring strategy: track system health, data quality, and model performance metrics, with automated alerts and a feedback loop. Emphasize proactive detection through statistical tests and business KPI correlation, and describe how you'd investigate and respond to degradation.

Pro tip: Tie model metrics to business outcomes (e.g., ticket routing accuracy, customer satisfaction) to prioritize alerts and avoid alert fatigue. Also, mention shadow deployment or canary testing for safe model updates.

1. Monitor System Health

Track infrastructure metrics (latency, throughput, error rates) and resource utilization to ensure the serving environment is stable. Set up alerts for anomalies that could impact model performance.

2. Track Data Quality

Monitor input data distributions, missing values, and schema changes to detect drift or pipeline issues. Use statistical tests (e.g., PSI, KL divergence) to compare production data against training data.

3. Measure Model Performance

Continuously evaluate model predictions using ground truth labels when available (e.g., delayed labels) and proxy metrics (e.g., confidence scores, prediction distribution). Set up dashboards and alerts for key metrics like accuracy, F1, and AUC.

4. Correlate with Business KPIs

Link model performance to business metrics (e.g., customer satisfaction, resolution time) to assess impact. Use these correlations to prioritize degradation alerts and communicate with stakeholders.

5. Implement Feedback Loops and Response

Establish a process for collecting feedback, investigating root causes (e.g., data drift, concept drift, bugs), and retraining or rolling back models. Automate where possible and document incidents for continuous improvement.

Key Points to Mention

  • Data drift detection (e.g., PSI, KL divergence, KS test)
  • Concept drift and delayed label handling
  • Model performance metrics (accuracy, F1, AUC, calibration)
  • Alerting and dashboarding (e.g., Prometheus, Grafana, custom dashboards)
  • Business KPI correlation (e.g., ticket routing accuracy, customer satisfaction)
  • Feedback loops and retraining triggers
  • Shadow deployment and canary testing for safe updates

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.