Description
We are trying to test the following AI features of codecept.io,
- ποΈββοΈ assist writing tests in pause() or interactive shell mode
- π self-heal failing tests (can be used on CI)
- π¬ send arbitrary prompts to AI provider from any tested page attaching its HTML contents
First 2 are working but when trying to use the third one (Arbitrary GPT Prompts), we are getting the error as 'I.askGptOnPageFragment is not a function'.
Note: Extended the AI helper in types file (we could also see the definitions for the methods 'askGptOnPage', 'askGptOnPageFragment').
Output from Terminal:
CodeceptJS v3.6.2 #StandWithUkraine
Using test root "C:\Users\hkrishn6\codecept_demo"
Run E-Commerce test on desktop --
Execution happening on desktop
Γ Place an order from demo e-commerce site on desktop @ai-test in 5304ms
-- FAILURES:
- Run E-Commerce test on desktop
Place an order from demo e-commerce site on desktop @ai-test:
I.askGptOnPageFragment is not a function
at Test. (tests\web_test.js:15:29)
Artifacts:
- screenshot: C:\Users\hkrishn6\codecept_demo\output\Place_an_o_1716549982.failed.png
FAIL | 0 passed, 1 failed // 6s
Run with --verbose flag to see complete NodeJS stacktrace
npm verb exit 1
npm verb code 1
npm verb exit 1
npm verb code 1
npm verb exit 1
npm verb code 1
Provide test source code if related
// paste test
-
CodeceptJS version: 3.6.2
-
NodeJS Version: 20.12.2
-
Operating System: Windows 11
-
puppeteer || webdriverio || testcafe version (if related) : Playwright : 1.41.1
-
Configuration file:
const {
setHeadlessWhen,
setCommonPlugins
} = require('@codeceptjs/configure');
require('./heal')
let aiLogs
// turn on headless mode when running with HEADLESS=true environment variable
// export HEADLESS=true && npx codeceptjs run
setHeadlessWhen(process.env.HEADLESS);
// enable all common plugins https://github.com/codeceptjs/configure#setcommonplugins
setCommonPlugins();
/** @type {CodeceptJS.MainConfig} */
exports.config = {
tests: './tests/web_test.js',
timeout: 260,
output: './output',
helpers: {
Playwright: {
url: "https://www.saucedemo.com",
show: true,
channel: 'msedge',
waitForAction: 100,
timeout: 180000
},
"Mochawesome": {
"uniqueScreenshotNames": "true"
}
},
ai: {
request: async (messages) => {
const allure = codeceptjs.container.plugins('allure');
const OpenAI = require('openai');
const openai = new OpenAI({
apiKey: "[key]"
})
const completion = await openai.chat.completions.create({
model: 'gpt-3.5-turbo',
messages
});
aiLogs = completion?.choices[0]?.message?.content
allure.createStep('AI self heal log', () => {
allure.addAttachment(
"heal_information.txt",
String(aiLogs),
"string"
);
});
return completion?.choices[0]?.message?.content;
}
},
plugins: {
allure: {
enabled: true,
require: '@codeceptjs/allure-legacy',
},
Playwright: {
coverage: {
enabled: true
},
trace: true,
video: true,
subtitles: {
enabled: true
}
},
heal: {
enabled: true
}
},
include: {
I: './steps_file.js'
},
}