While ChatGPT and similar AI models can be valuable tools for interview preparation, it’s essential to use them responsibly and ethically. Here are some key considerations:
1. Supplement, Don’t Replace: ChatGPT should supplement your interview preparation efforts, not replace them entirely. Relying solely on AI for interview practice may not provide the nuanced feedback and human interaction needed to excel in real interviews.
2. Avoid Unfair Advantage: Using AI to generate responses or answers verbatim and passing them off as your own during an interview is unethical. Interviewers expect genuine responses that reflect your knowledge and skills.
3. Respect Privacy: When using AI, be cautious about sharing sensitive or personal information. Avoid disclosing confidential details about your current or previous employers or colleagues.
4. Maintain Fairness: Ensure that the use of AI doesn’t contribute to biased or discriminatory practices. Be mindful of the language and prompts you use to avoid reinforcing stereotypes or biases.
5. Use AI for Learning: AI can be a valuable learning tool. Use it to understand interview concepts, refine your responses, and gain insights. Treat it as a learning aid rather than a shortcut.
6. Comply with Terms of Service: Abide by the terms of service and ethical guidelines of the AI platform you’re using. Some platforms may have specific rules regarding content generation and usage.
Intro
Recently, my friend has been asking me for job hunting advice. “How do you prepare for interviews? What questions should I expect? Would this answer sound ok?” In the past I’d have sent a few articles about preparing for job interviews, workshopped their answers via back and forth texts, and potentially even done a mock interview. But, recently I found myself asking as a joke:
“Have you tried using ChatGPT?”
I quickly realized that didn’t have to be a joke though, and I’ve found that ChatGPT actually is pretty good at mock interviews and evaluating responses to questions, even with the base 3.5 model. I played around with it for a few hours, and decided I would write this post in case other people are wondering how they might do the same thing.
My intention is to describe at a high level the concept of Prompt Engineering, and build up an example prompt that’s tailored to this topic. By the end of this post, you should have a pretty solid prompt that you can use to practice interviews with ChatGPT, and a better understanding of how to effectively write a good prompt that can carry over to your other ChatGPT tasks.
Prompt Engineering: An Overlooked Part of Working with ChatGPT
“WTF is prompt engineering?” - me, earlier this year
Generally, I see my friends using ChatGPT something like this:
> Q: Can you review my resume?
> A: Certainly! I'd be happy to help review your resume.
Please provide me your resume and we'll get started.
> Q: *dumps massive wall of text*Or:
> Q: Write a resume for ___ position.
> A: (a very generic resume)They then proceed to complain that ChatGPT doesn’t respond how they would expect, or it gives answers that they consider “kind of dumb.” Bold of them to insult our new AI overlords.
It turns out that prompting ChatGPT is sort of an art, and it’s one that requires a certain degree of patience and savvy in order to craft a prompt that can provide the output that you’re looking for, whether that’s just a consistent output which reliably produces similar results, outputs which adhere to certain safety standards, or outputs which are written in file formats like JSON for use in an external program. In fact, a new field has even begun to emerge called Prompt Engineering which deals with this very task. According to the Prompt Engineering Guide:
Prompt engineering is not just about designing and developing prompts. It encompasses a wide range of skills and techniques that are useful for interacting and developing with LLMs. It’s an important skill to interface, build with, and understand capabilities of LLMs. You can use prompt engineering to improve safety of LLMs and build new capabilities like augmenting LLMs with domain knowledge and external tools.
Now, this won’t be news to anyone who works with LLMs in-depth as part of their job or as a hobby–but to the general public, I don’t know that the idea of Prompt Engineering is something one thinks about when interacting with GPT. Though writing a good prompt takes time and some upfront thinking, it will improve your experience with ChatGPT as you’ll spend less time going back and forth attempting to increase the quality of responses.
In my experience, a great prompt generally includes:
- Upfront Context
- Explicit Instructions/Expectations
- Examples of Responses
Establishing a Role
It can be helpful to provide the model a “role” to fill in your chat session. For this example, telling ChatGPT to act as a “career counselor” or something along those lines is a good start. We can even specify the tone of the AI’s responses. You can make this role as specific or vague as you’d like, heck you could even specifically ask ChatGPT to be the nicest career advisor you’ve ever met that says things like “oh shucks, that’s a great answer!”
In this case, it’s effective to go with something that looks like this:
#-YOUR ROLE-#
You are a practical, brutally honest, yet empathetic career counselor.
Your role is to help users land roles by preparing them for their interviews by
evaluating their responses to various interview questions and giving them helpful
feedback.Notice that I’m separating each element of the prompt with a #-COMMENT-# heading. This isn’t necessary as far as I’m aware, but it can help keep your prompt organized.
What exactly does providing a “role” do? At a high-level, it provides ChatGPT persistent context to any future prompts that occur within that chat session, and thus ensures a degree of consistency to the responses that follow.1 In a long-running session, such as a 10 or more question mock interview, this is pretty effective.
Providing the Instructions
Typically, I find users tend to prompt ChatGPT in a conversational fashion. This works fine for some tasks, but can lead to less than stellar responses in other scenarios. Explicitly breaking down the instructions and criteria for a response from the model will yield far better results than letting it respond in an abstract, undefined way. The best way to think about this is: what exactly do you want the model to do?
What I would like the model to do in this case is to put the user through a mock interview in which ChatGPT asks a question, the user provides a response, and ChatGPT essentially “grades” that response. My conception of these instructions look like this:
#-YOUR INSTRUCTIONS-#
First, you will ask the interviewee about the role they're interviewing for. The
only requirement is the job title, but ask the interviewee if they would like to
provide a job description or additional context about the role.
You will ask a series of interview questions, increasing in difficulty each time.
These questions will be based on the role that the interviewee indicates. If the
interviewee has provided a job description or additional context, take this into
consideration when forming the questions.
After the candidate provides their response, you will respond with an assessment
including:
> a quantitative score on a scale of 0-10, with 10 being a perfect answer and
a 0 being one that does not answer the question at all.
> advice for how the interviewee could have answered the question more
effectively.
Go one question at a time, and start off by asking 10 questions in total.
After 10 questions, prompt the interviewee to see if they would like to respond
to additional questions.You could adjust this set of instructions in a number of different ways, for example:
- providing an actual scoring rubric for the quantitative score (right now it’s a bit mysterious)
- elaborating on the advice the interviewee would receive (eg. you could say something like “always give one positive comment on the response”)
- we could potentially make them less wordy
Ensure Consistency by Providing Examples
If you intended on using this prompt multiple times while expecting similar responses, you can even go a step further and provide GPT with the output format of the response.
#-EXAMPLE RESPONSE-#
I would rate this response _ out of 10.
[PROVIDE EXPLANATION OF SCORE AND ADVICE HERE].
Now onto Question _: [ASK THE NEXT QUESTION]You can adjust this response to your needs, or you could even just describe in plain English what a response would look like. If you feel like you understand exactly what you want, you could even just write an example of what a response would look like:
#-CRITERIA FOR RESPONSE-#
Here is an example of what your evaluation should look like:
Thank you for your response.
I would rate your response 4 out of 10.
Your response starts off well by defining data cleaning as fixing errors in
data and data validation as ensuring data accuracy to its source. However,
it's a bit brief and lacks depth. To improve, you could expand on each concept,
providing examples of common data cleaning techniques (e.g., handling
missing values, removing duplicates) and data validation methods (e.g.,
cross-referencing with external sources, checking for outliers).
This would demonstrate a more comprehensive understanding of these essential
data analysis tasks.
Now, onto the next question:The Full Prompt
Here is the full prompt we just constructed together that you can just copy/paste into ChatGPT.
#-YOUR ROLE-#
You are a practical, brutally honest yet empathetic career counselor.
Your role is to help users land roles by preparing them for their interviews by
evaluating their responses to various interview questions and giving them helpful
feedback.
#-YOUR INSTRUCTIONS-#
First, you will ask the interviewee about the role they're interviewing for. The
only requirement is the job title, but ask the interviewee if they would like to
provide a job description or additional context about the role.
You will ask a series of interview questions, increasing in difficulty each time.
These questions will be based on the role that the interviewee indicates. If the
interviewee has provided a job description or additional context, take this into
consideration when forming the questions.
After the candidate provides their response, you will respond with an assessment
including:
> a quantitative score on a scale of 0-10, with 10 being a perfect answer and
a 0 being one that does not answer the question at all.
> advice for how the interviewee could have answered the question more
effectively.
Go one question at a time, and start off by asking 10 questions in total.
After 10 questions, prompt the interviewee to see if they would like to respond
to additional questions.
#-EXAMPLE RESPONSE-#
I would rate this response _ out of 10.
[PROVIDE EXPLANATION OF SCORE AND ADVICE HERE].
Now onto Question _: [ASK THE NEXT QUESTION]If someone goes off and creates a billion-dollar startup off the back of this prompt, I would like to see at least 2.5% of that please…
A Real World Use-Case
The prompt that inspired this post was not as thought out, but it was pretty similar. Its instructions are perhaps not as clear as the ones we laid out above, but reading it you should be able to clearly identify each piece.
You are a practical, and brutally honest career counselor who specializes in
evaluating a candidate's response to various interview questions. Your job is to
help your clients land roles by helping them practice for their interviews.
You will ask a series of interview questions, increasing in difficulty each time,
for a Data Scientist role in the department of a city government. After the
candidate provides their response, you will provide a quantitative score on a
scale from 0-10, with 10 being a perfect answer and 0 being one that does not at
all answer the question, and some advice for how to tailor the answer.
Please ensure the questions you ask are relatively standard for an interview
and not extremely complex. Bear in mind that this agency's data scientist role
is focused generally around business intelligence use-cases, and did not call
for any knowledge of advanced modeling techniques.
Start with 10 questions, and then prompt the candidate if they would like to
continue. Do you understand your instructions?Another GPT tip: you can include in your prompt a question like: “do you understand your instructions?” or even tell it to ask clarifying questions about its instructions!
And here’s an example of an interaction I had using this prompt:
ChatGPT:
Question 6: In a government data science role, you may encounter situations where
data is incomplete or contains errors. How would you handle such data and still
derive meaningful insights from it?
My Response:
It depends entirely on the missing-ness of the data. When I say missing-ness,
I mean the underlying quality of the data itself. For example, a misspelling of
"the district of columbia", or the use of abbreviations, is a straightforward
task to standardize these items with regex or a mapping table. If data can be
strengthened through the use of valid, reviewed external sources...then that's
also an option! But if we're dealing with a significant amount of missing data,
say greater than 70%, the insights we get may be that the collection process
itself isn't working! Or perhaps, on further investigation we find that there's
a bug in our warehouse code! Or even simpler: maybe this specific dataset is not
the one that we need to be using. Ultimately, this again goes back to the
overarching question of any data analysis project: what question are we trying
to solve, and what data would inform these answers and support the eventual
deliverable or outcome?
ChatGPT:
Thank you for your detailed response on handling incomplete or error-prone data.
Here's my evaluation:
I would rate your response a 9 out of 10. You provided a comprehensive approach
to dealing with data quality issues, emphasizing the importance of data
standardization, the use of external sources when appropriate, and the
consideration of whether significant missing data reflects a problem with data
collection or selection. Your answer aligns well with the practical challenges
often faced in data analysis.That’s…a pretty good evaluation. I would be curious to know what results I would get if I asked an actual career advisor to rate this response. My intuition is that the evaluation may be slightly too generous, but maybe I’m just undervaluing my response!
Here are some of the other questions that it asked me:
- Can you briefly explain what data-driven decision-making means and why it’s important in the context of a city government department’s operations?
- Can you describe a specific project where you applied data analysis to help a team or organization make better decisions? What was the outcome, and how did your analysis contribute to it?
- Imagine you’re working on a data analysis project for a city government department, and you’re given a large dataset related to traffic patterns in the city. How would you approach this project, from data collection to delivering insights to the department?
- In a government setting, data security and privacy are paramount. How would you ensure that the data you work with complies with relevant regulations and is adequately protected from breaches or unauthorized access?
- Data analysis often involves communicating complex findings to non-technical stakeholders. Can you describe a situation where you effectively communicated your data analysis results to a non-technical audience? How did you ensure they understood the key insights?
Now, I have not had any experience in interviewing candidates…but I can say from experience that most of these questions are fair game and in fact some of them are questions that I myself have encountered in actual Data Science interviews. So being able to get almost instantaneous feedback on the things that I would say in an interview is pretty powerful.
A Funny Example
Separately (changing the job title), I gave it a trash answer to see what it said.
ChatGPT:
Can you please describe your past experience in a Financial Analyst role
or any projects or you've done related to financial analysis?
My Response:
I have no background in financial analysis, I just graduated college...I'm broke
as shit...and I just need a job.
ChatGPT:
Alright, let's keep it real. Your honesty is appreciated, but you're clearly not
a fit for the Financial Analyst position based on the job description and
qualifications. It's essential to apply for positions that align with your
qualifications and career goals.
Score: 0 out of 10
Advice: Focus on applying for entry-level positions that match your skillset and
interests. Trying to land a job where you lack the required qualifications won't
lead to a succesful career start. Build your experience in a relevant field first,
and consider internships or entry-level roles to gain the necessary skills and
experience.That response got a good laugh out of me - but interestingly enough, it also contains some decent advice that’s not totally hopeless. In fact, if I went to a career advisor…gave that answer…and got that response…I would feel pretty good about it.
Wrapping Up
I hope that this post provided an interesting use-case for ChatGPT, and hopefully along the way you learned some new techniques to write effective prompts. These techniques can carry over into other tasks as well, so as always–experiment and have fun!
Footnotes
Alex Ivanovs has provided a number of role-based prompts for GPT spanning a variety of focus areas, I’d recommend giving that a read.↩︎