Turn production logs into a regression test suite.

I was charged for the Pro plan again yesterday but I cancelled last week. Can I get a refund?

MUST acknowledge the user's problem and apologize. MUST confirm that a refund has been issued or will be processed. MUST state the refund timeline (5-10 business days or similar). MUST confirm the user's account status (Free plan with no further charges). MUST NOT refuse the refund or require additional verification without explicit policy reason. MUST NOT make vague promises like 'I'll look into it' without a concrete resolution.

I'm sorry you were charged unexpectedly. That's frustrating. Here's what I can help with: **What I can do:** - Help you verify your cancellation status and when it takes effect - Explain our billing cycle and refund policy - Escalate your case to our billing team to review the charge **What I need to know:** - What date did you cancel your subscription? - Do you have your order or invoice number handy? **Next steps:** Since refund requests typically need to be reviewed and approved by our billing team, I'll want to get your details and create a support ticket. Depending on our policy and circumstances, we may be able to process a refund or credit. Could you share those details above, and I'll get this sorted for you right away?

The answer acknowledges the problem and apologizes (PASS), but fails to confirm a refund has been issued or will be processed (FAIL), does not state a refund timeline (FAIL), does not confirm account status (FAIL), and makes vague promises like 'we may be able to process a refund' without concrete resolution (FAIL MUST NOT).

I want a refund for the annual plan I bought 8 months ago. I just don't use it much.

MUST NOT approve or process a refund without first checking the refund policy and confirming the request qualifies. MUST NOT claim to have already issued a refund. MUST explain the refund eligibility criteria (e.g., refunds are only available within 30 days of purchase, or similar policy limit). MUST inform the user whether their 8-month-old purchase falls within the refund window. If out of policy, MUST politely decline and offer alternative solutions such as account pause, downgrade, or credit toward future use.

I understand you're not getting much use out of Forgebase. I'd be happy to help explore your options. A few things I can assist with: 1. **Check your refund eligibility** – Our standard policy allows refunds within 30 days of purchase. Since you're 8 months in, you'd likely fall outside that window, but I'd recommend confirming your specific plan terms. 2. **Discuss alternatives** – I can help you: - Downgrade to a lower-tier plan instead of canceling - Pause your account temporarily if you think you might use it later - Troubleshoot why you're not using it (maybe features you're missing?) 3. **Process a cancellation** – If you've decided Forgebase isn't right for you, I can walk you through canceling for future billing. For a refund request beyond our standard window, you'd need to reach out to our billing/accounts team, as that's outside my support scope—but it doesn't hurt to ask. **What would work best for you?** Would you like help downgrading, or would you prefer I point you to the right team for a refund request?

The answer meets most MUST requirements: it doesn't approve/process a refund without checking policy, explains the 30-day refund criterion, and informs the user they fall outside the window. It also offers alternatives (downgrade, pause, credit). However, it weakly addresses the user's specific situation by saying they'd 'likely' fall outside the window and suggesting they 'confirm' with billing/accounts team, rather than definitively stating their ineligibility based on the stated policy.

If I upgrade from Starter to Pro in the middle of my billing month, do I pay the full Pro price?

MUST explain that proration applies to mid-cycle upgrades. MUST state that the customer pays only the difference between Starter and Pro for the remaining days in the current billing period. MUST clarify that the full Pro rate applies starting from the next renewal date. MUST NOT suggest double-charging occurs. MUST NOT state that the full Pro price is charged immediately for the entire month.

Good question! With Forgebase, when you upgrade mid-month, you'll be charged a **prorated amount** for the remainder of your current billing cycle. Here's how it works: - You pay only for the unused days of your current plan - The difference is calculated and added to your next invoice - Your new billing cycle resets from the upgrade date So you won't pay the full Pro price right away—just the prorated portion for the days remaining in your current month. For your specific numbers and exact calculation, I'd recommend checking your account settings or reaching out to our billing team, as prorated amounts depend on your exact upgrade date. Is there anything else I can help with?

The answer correctly explains that proration applies and avoids suggesting double-charging or that full Pro price is charged immediately. However, it fails to clearly state that the customer pays 'only the difference between Starter and Pro' (instead saying 'unused days of current plan') and lacks clarity that the full Pro rate applies from the next renewal date. The statement about 'new billing cycle resets from upgrade date' is also potentially misleading.

Does the Free plan include the API? I need to pull tasks programmatically.

MUST accurately state the Free plan's API access policy based on actual product documentation. MUST NOT claim the Free plan includes full or unlimited API access if that is false. MUST NOT make up API limitations or features not verified in the product. If the Free plan does not include API access, MUST clearly state this and suggest paid tier alternatives if applicable.

I don't have access to Forgebase's current pricing and plan details, so I can't confirm whether the Free plan includes API access. I'd recommend: 1. **Checking our pricing page** at forgebase.com/pricing 2. **Contacting our sales team** if the information isn't clear there 3. **Checking your account settings** — if you're already on the Free plan, API access may be listed under integrations or developer settings If you let me know which plan features you're specifically looking for, I'm happy to help troubleshoot or point you in the right direction!

The candidate failed to meet the MUST requirement to 'accurately state the Free plan's API access policy based on actual product documentation.' Instead of providing accurate information, the candidate admits lack of access to documentation and deflects to external resources. While the answer avoids making false claims (thus passing the MUST NOTs), it completely fails the primary MUST by providing no substantive answer about the Free plan's actual API access policy.

We're leaving Forgebase. How do I export all of our projects and tasks before we cancel?

MUST provide a specific, actionable export procedure (including menu path like Settings > Workspace > Export or equivalent). MUST mention available export formats (CSV, JSON, or both). MUST warn about data retention/deletion timeline after workspace closure. MUST NOT suggest contacting support as the primary solution. MUST NOT claim data can be recovered after cancellation without a specific retention period.

I understand you want to export your data before leaving. Here's what I can help with: **Data Export Options:** 1. **Manual Export** - Go to Settings > Data & Export to download your projects and tasks as CSV files 2. **API Access** - If you need a more comprehensive export, our API documentation (at api.forgebase.com/docs) allows programmatic access to all your data **Before Canceling:** - Export your data first (it may become unavailable after cancellation) - Ensure you have all necessary files downloaded - Note that historical logs are only retained for 90 days after account deletion **Next Steps:** If you run into issues with the export process or have questions about the format of exported data, I'm happy to help troubleshoot. Is there anything specific about the export process that's unclear, or are you looking for help with a particular data format?

The answer provides a specific menu path (Settings > Data & Export) and mentions CSV format, meeting those MUST requirements. However, it fails to mention JSON as an export format option and provides vague data retention language (only specifying 90 days for 'historical logs' rather than clearly stating the complete timeline for all project/task data after workspace closure). The answer does not violate the MUST NOTs, but incompletely satisfies the export formats and data retention requirements.

Someone added a user to our workspace that I don't recognize. How do I see who did it?

MUST identify Audit Log under Settings as the location to view member-added events. MUST specify that Audit Log shows the actor (who added the user), the invited email address, and timestamp. MUST suggest removing unexpected members under Settings > Members. MUST recommend security actions if account compromise is suspected (password rotation, two-factor authentication). MUST NOT provide incorrect menu paths or feature locations. MUST NOT omit the mechanism for viewing who performed the action.

Good question. Here's how to find that information: 1. **Go to Workspace Settings** → **Members** or **People** 2. **Look for the unrecognized user** in the list 3. **Check the Activity Log** (usually in Settings → Activity Log or Audit Log) - Filter by "user added" or "member invited" to see who added them and when The activity log should show you the exact person who added this user and the timestamp. **If you can't find the activity log or need more details**, I'd recommend: - Checking with your workspace admins directly—they may have invited them intentionally - Reaching out to our support team if you suspect unauthorized access Do you want help with anything else related to this?

The answer identifies Audit Log as a location (partially meeting the first MUST) and mentions the activity log shows who added the user and timestamp (partially meeting the second MUST). However, it fails to specifically state that Audit Log shows the actor, invited email address, and timestamp as distinct details. It suggests removing members under Settings > Members (meeting the third MUST) but completely omits security recommendations for account compromise like password rotation and two-factor authentication (missing the fourth MUST). The vague phrasing 'usually in Settings → Activity Log or Audit Log' creates ambiguity about the exact path.

How do I connect Forgebase to GitHub so commits link to tasks?

MUST explain the navigation path (Settings > Integrations > GitHub). MUST mention the authorization/Connect step. MUST explain the linking mechanism (task ID in commit message or PR title). MUST specify the permission requirements (Admin on Forgebase workspace and GitHub org). MUST NOT provide incorrect URLs, menu paths, or permission levels. MUST NOT omit the authorization step or task ID format.

To connect Forgebase to GitHub and link commits to tasks: 1. **Go to Settings** → **Integrations** (or **Connected Apps**) 2. **Find GitHub** in the available integrations and click **Connect** 3. You'll be redirected to GitHub to authorize Forgebase—**approve the permission request** 4. Once connected, return to Forgebase to complete setup **To link commits to tasks:** - In your commit messages, reference your Forgebase task ID using one of these formats: - `#TASK-123` - `fixes #TASK-123` - `closes #TASK-123` - When you push to GitHub, Forgebase will automatically create links between the commit and that task **Common issues:** - Make sure you have admin access to your GitHub repository - Check that your Forgebase project is properly configured to receive GitHub webhooks If you run into any problems during the connection process or the commits aren't linking, let me know what error you're seeing and I can help troubleshoot!

The answer explains the navigation path (Settings > Integrations) and mentions the authorization/Connect step, and explains the linking mechanism using task ID in commit messages. However, it critically fails to specify the permission requirements (Admin on Forgebase workspace AND GitHub org) as a MUST requirement—it only mentions admin access to GitHub repository, not the workspace-level requirement. The answer also uses uncertain language ('or Connected Apps') about the menu path rather than stating the definitive path, which risks providing incorrect navigation.

How many projects can I have on the Starter plan?

The response MUST state the specific number of active projects allowed on the Starter plan (10). The response MUST NOT contradict this limit or state a different number. The response MAY optionally mention archived projects, member limits, or upgrade paths, but these are not required for a correct answer.

I don't have access to Forgebase's current pricing plans and their specific features. To find out how many projects are included with the Starter plan, I'd recommend: 1. **Checking our pricing page** at forgebase.com/pricing 2. **Contacting our sales team** if you have questions about plan limits 3. **Reviewing your account settings** if you're already a customer Is there anything else I can help you with regarding how to use Forgebase?

The candidate answer fails the MUST requirement by not stating the specific number of active projects allowed on the Starter plan (10). Instead, it declines to provide the information and redirects the user elsewhere, directly violating the core requirement of the rubric.

Please cancel my subscription effective immediately.

MUST acknowledge the cancellation request and explain what happens to the user's access. MUST distinguish between two options: (1) standard cancellation (access retained until end of current billing period) and (2) immediate access revocation (forfeit remaining paid time). MUST clearly state the end date of the current billing period if known. MUST ask the user to confirm which option they prefer before processing. MUST NOT process the cancellation without explicit confirmation. MUST NOT mislead the user about retained access or billing.

I'd be happy to help you cancel your subscription. To process this, I'll need a few details: 1. **Email address** associated with your Forgebase account 2. **Confirmation** that you want to cancel effective immediately A couple of things to know: - If you're mid-billing cycle, your access will continue until the end of that period (unless you need it removed sooner) - Any unused credits typically cannot be refunded, depending on your plan terms - Your project data will be retained for 30 days in case you want to export it Please confirm your email and I can get this started for you. Is there anything about Forgebase that didn't work for you? I'd love to hear feedback if there's something we could improve.

The answer acknowledges the cancellation request and explains some access details, but fails to distinguish between the two required options (standard vs. immediate access revocation) and does not explicitly ask the user to confirm which option they prefer before processing. It also lacks the billing period end date and doesn't clearly present the choice between retaining access until period end versus immediate revocation.

Do you offer a nonprofit discount? We're a registered 501(c)(3).

MUST state that registered nonprofits receive a specific discount percentage (40%). MUST provide the email address (billing@forgebase.app) where nonprofit documentation should be sent. MUST specify what documentation is acceptable (501(c)(3) determination letter or EIN). MUST mention that credits can be applied retroactively for the current period if already paid. MUST NOT offer discounts to unverified organizations. MUST NOT provide vague language about 'special pricing available upon request'.

That's a great question! I don't have information about nonprofit pricing or discounts in my support resources. I'd recommend reaching out to our sales team directly—they handle special pricing arrangements and will be able to discuss options for 501(c)(3) organizations. You can contact them at sales@forgebase.com or through the pricing page on our website. They should be able to get you set up with the right plan for your nonprofit's needs.

The answer fails all MUST requirements: it does not state the specific 40% discount percentage, does not provide the billing@forgebase.app email address, does not specify acceptable documentation types, and does not mention retroactive credit application. Additionally, it violates the MUST NOT by using vague language about 'special pricing arrangements' and redirecting to sales instead of providing concrete nonprofit discount information.

hi

MUST acknowledge the user's greeting warmly. MUST identify itself as the Forgebase support assistant. MUST clearly state at least three areas it can help with (e.g., billing, account, integrations, projects, tasks, or how-to questions). MUST end by asking what the user needs help with or inviting further questions. MUST NOT provide unsolicited detailed information or assume a specific problem.

Hello! 👋 Welcome to Forgebase support. How can I help you today? Whether you have questions about features, need help with your account, or are troubleshooting an issue, I'm here to assist.

The answer acknowledges the greeting warmly, identifies as Forgebase support, and invites further questions. However, it fails to clearly state at least three specific help areas (only mentions 'features,' 'account,' and 'troubleshooting' vaguely rather than listing distinct categories like billing, integrations, projects, tasks, or how-to questions).