fix(oauth): list ChatGPT models as Codex CLI 0.160.0 by devin-ai-integration[bot] · Pull Request #760 · tester-army/e2e

devin-ai-integration · 2026-10-02T10:29:33Z

Closes #752.

What and why

listCodexModels sends client_version to the ChatGPT backend's /models, and the backend leaves out models newer than that client. The constant was 0.155.1; the current @openai/codex on npm is 0.160.0, so e2e models openai missed models newer Codex CLIs already see.
Bumps CODEX_CLIENT_VERSION in packages/e2e/src/oauth/providers/openai.ts to 0.160.0. Nothing else reads it.
Not in this PR: automating the bump (the issue's second option). Happy to add a scheduled check if wanted.

Verified

Ran it locally: no - needs a ChatGPT login, which this environment does not have. npm view @openai/codex version returns 0.160.0; tests/unit/oauth passes (it pins the client_version format, not the value).

Checklist

Changeset added (pnpm changeset) for any change to a published package, or the change is testbed, docs, or CI only.
Breaking change: title carries !, the changeset body names what breaks and what replaces it, and the deprecation warning landed a window earlier (stability policy in CONTRIBUTING.md).
Docs updated in the same PR: the docs/**/*.mdx page for the behavior, and skills/e2e/ if the skill describes it.
Contract change: emitted .d.ts reviewed, tests/types/sdk-types.ts updated; wire change edits the schema and both fixtures.
Engine contract change (e2e/engine): changesets for e2e, @e2e-dev/web, and @e2e-dev/mobile.

Link to Devin session: https://app.devin.ai/sessions/10c70735b1d647a0913672025b846b61
Open in Devin Desktop: https://app.devin.ai/desktop/session/10c70735b1d647a0913672025b846b61?variant=devin
Requested by: @okwasniewski

devin-ai-integration · 2026-10-02T10:29:36Z

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

Disable automatic comment, CI, and merge conflict monitoring

github-actions · 2026-10-02T10:32:30Z

🟢 e2e web: 90 passed

All 90 tests in 41 files

	Test	Time
🟢	tests/aria-hidden-flow.e2e.ts · 2 passed	458ms
🟢	aria hidden flow › picks the Pro plan by its visible text and confirms it	313ms
🟢	aria hidden flow › another plan shows the error and keeps the picker open	145ms
🟢	tests/async-states.e2e.ts · 2 passed	7.1s
🟢	async states › reads the code off the transient toast and verifies it once the button enables	2.3s
🟢	async states › a wrong code is rejected after the toast has gone	4.8s
🟢	tests/canvas-only.e2e.ts · 2 passed	242ms
🟢	canvas only › taps square, circle, then diamond at their pixel coordinates	136ms
🟢	canvas only › a shape out of order paints the error and resets progress	106ms
🟢	tests/canvas-whiteboard.e2e.ts · 2 passed	346ms
🟢	canvas whiteboard › drags both shapes into their slots and saves	178ms
🟢	canvas whiteboard › bounces a shape off the wrong slot and refuses to save early	168ms
🟢	tests/cart-totals.e2e.ts · 2 passed	327ms
🟢	cart totals › increasing a quantity updates the line total but the order summary stays stale (planted bug)	187ms
🟢	cart totals › a quantity never drops below one	140ms
🟢	tests/checkout-review.e2e.ts · 1 passed	162ms
🟢	shows a raw greeting token and an undefined delivery date (planted bug), yet places the order	162ms
🟢	tests/context-menu-trap.e2e.ts · 2 passed	522ms
🟢	context menu trap › right-click opens nothing; the row menu renames the file	268ms
🟢	context menu trap › the decoy menu items refuse and keep the file	254ms
🟢	tests/control-inventory.e2e.ts · 7 passed	3.1s
🟢	control inventory › check sets the checkbox and the radio without flipping a set one	220ms
🟢	control inventory › a long press and a double click are told apart from a tap	967ms
🟢	control inventory › the file menu opens on right-click only and Rename renames the file	239ms
🟢	control inventory › setInputFiles attaches files named from the project root	115ms
🟢	control inventory › the details view is left through the browser history	143ms
🟢	control inventory › scrollIntoView brings the footnote into view	237ms
🟢	control inventory › every exercise done shows the success message	1.2s
🟢	tests/css-content-ui.e2e.ts · 2 passed	346ms
🟢	css content ui › the tile labelled Ship it advances and confirming ships	206ms
🟢	css content ui › a wrong tile shows the CSS-rendered error and keeps the picker	140ms
🟢	tests/date-picker.e2e.ts · 2 passed	689ms
🟢	date picker › navigates three months ahead and books the required date	378ms
🟢	date picker › confirming without the exact date shows the error	311ms
🟢	tests/debounced-search.e2e.ts · 2 passed	2.9s
🟢	debounced search › a decoy one character off is refused	1.4s
🟢	debounced search › picking the exact item adds it to the cart	1.4s
🟢	tests/deferred-dialog.e2e.ts · 1 passed	1.1s
🟢	one submit opens one confirm, even when double-clicked	1.1s
🟢	tests/div-soup.e2e.ts · 2 passed	568ms
🟢	div soup › a wrong code is rejected	249ms
🟢	div soup › the right code grants access after the delete tile fixes a typo	319ms
🟢	tests/drag-and-drop.e2e.ts · 2 passed	667ms
🟢	drag and drop › an out-of-order item resets the dropzone	318ms
🟢	drag and drop › dropping Banana then Cherry passes	349ms
🟢	tests/filter-deep-link.e2e.ts · 2 passed	274ms
🟢	filter deep link › filtering writes the query to the URL	156ms
🟢	filter deep link › opening a shared link restores the filter	118ms
🟢	tests/gift-card-purchase.e2e.ts · 3 passed	554ms
🟢	gift card purchase › refuses to buy without an amount	153ms
🟢	gift card purchase › refuses a recipient without an at sign	192ms
🟢	gift card purchase › sends the gift card	209ms
🟢	tests/hover-menu.e2e.ts · 2 passed	485ms
🟢	hover menu › a decoy leaf is refused	182ms
🟢	hover menu › hovering Account, then Billing, reaches Redeem voucher	303ms
🟢	tests/iframe-form.e2e.ts · 2 passed	436ms
🟢	iframe form › a wrong code is refused by the host page	205ms
🟢	iframe form › the code read from one frame applies through the other	231ms
🟢	tests/image-only-ui.e2e.ts · 3 passed	449ms
🟢	image only ui › the icons have no names, so the bare svg locator is ambiguous	106ms
🟢	image only ui › a wrong icon shows the error image and keeps the icons	149ms
🟢	image only ui › tapping the star, then the heart, verifies the icons	194ms
🟢	tests/infinite-scroll.e2e.ts · 1 passed	4.0s
🟢	scrolls the feed until the Golden Ticket loads, then claims it	4.0s
🟢	tests/kanban-board.e2e.ts · 3 passed	716ms
🟢	kanban board › pointer drags move both cards into the goal layout	383ms
🟢	kanban board › a swipe along a path drags a card as well	149ms
🟢	kanban board › a plain tap moves nothing and the untouched board is rejected	184ms
🟢	tests/login-form.e2e.ts · 4 passed	666ms
🟢	login form › rejects a malformed email	165ms
🟢	login form › rejects a short password	147ms
🟢	login form › rejects an unknown account	152ms
🟢	login form › signs in and logs out	202ms
🟢	tests/lying-labels.e2e.ts · 4 passed	562ms
🟢	lying labels › the button whose accessible name says Pay now is the wrong one	130ms
🟢	lying labels › a text match for Pay now catches the decoy phrase too	89ms
🟢	lying labels › the field whose accessible name says Amount is the wrong one	176ms
🟢	lying labels › visible text drives the payment through	167ms
🟢	tests/native-dialogs.e2e.ts · 3 passed	348ms
🟢	native dialogs › answers the confirm, prompt, and alert chain	118ms
🟢	native dialogs › dismissing the confirm cancels the order	114ms
🟢	native dialogs › a wrong gift code is rejected	116ms
🟢	tests/newsletter-signup.e2e.ts · 1 passed	2.2s
🟢	subscribe renders no feedback and logs a TypeError (planted bug)	2.2s
🟢	tests/onboarding-wizard.e2e.ts · 5 passed	4.3s
🟢	onboarding wizard › back keeps what was typed and the brief comes back with it	293ms
🟢	onboarding wizard › the address step takes the brief through its bare-div dropdown	292ms
🟢	onboarding wizard › the profile step steers the calendar three decades back	1.1s
🟢	onboarding wizard › the preferences step flips the toggles the brief names and leaves the trap off	1.3s
🟢	onboarding wizard › the review step repeats the brief and finishes with the PIN	1.3s
🟢	tests/order-history.e2e.ts · 2 passed	1.4s
🟢	order history › overview and settings render their panels	177ms
🟢	order history › the orders tab never leaves its loading state (planted bug)	1.2s
🟢	tests/otp-auto-advance.e2e.ts · 3 passed	1.4s
🟢	otp auto-advance › focus advances with each digit and the full code verifies	451ms
🟢	otp auto-advance › backspace on an empty box clears and refocuses the previous one	129ms
🟢	otp auto-advance › a wrong code clears every box and starts over	784ms
🟢	tests/overlay-trap.e2e.ts · 1 passed	717ms
🟢	continues only after the banner and the overlay are gone	717ms
🟢	tests/playbook-cleanup.e2e.ts · 2 passed	429ms
🟢	playbook cleanup › deleting the draft removes it from an exact-name search	213ms
🟢	playbook cleanup › cancelling the dialog keeps the draft searchable	216ms
🟢	tests/price-sorting.e2e.ts · 3 passed	343ms
🟢	price sorting › opens in the featured order with sale prices struck through	104ms
🟢	price sorting › low to high sorts by the original price, not the paid one (planted bug)	110ms
🟢	price sorting › high to low is the same wrong order reversed (planted bug)	129ms
🟢	tests/product-reviews.e2e.ts · 1 passed	1.2s
🟢	the running-shoe page lists coffee-grinder reviews (planted bug)	1.2s
🟢	tests/promo-storefront.e2e.ts · 1 passed	1.4s
🟢	buys a deal once the promo chrome is out of the way	1.4s
🟢	tests/rich-text-editor.e2e.ts · 2 passed	409ms
🟢	rich text editor › bolding exactly the required word passes the check	211ms
🟢	rich text editor › bold that spills into the previous word fails the check	198ms
🟢	tests/saved-addresses.e2e.ts · 3 passed	501ms
🟢	saved addresses › adds an address and clears the form	164ms
🟢	saved addresses › ignores an add with an empty field	157ms
🟢	saved addresses › delete shows its toast but keeps the address, on retry too (planted bug)	180ms
🟢	tests/shadow-dom-form.e2e.ts · 2 passed	502ms
🟢	shadow DOM form › rejects a wrong access code	249ms
🟢	shadow DOM form › grants access with the code from the hint inside the root	253ms
🟢	tests/smoke.e2e.ts · 1 passed	404ms
🟢	home lists the scenarios and opens one	404ms
🟢	tests/stale-dom.e2e.ts · 1 passed	2.2s
🟢	taps the moving target three times across rebuilds of the list	2.2s
🟢	tests/sticky-chrome.e2e.ts · 1 passed	185ms
🟢	accepts the terms once the button clears the sticky footer	185ms
🟢	tests/virtualized-table.e2e.ts · 1 passed	1m 5s
🟢	scrolls the windowed table to the Golden Row and claims it	1m 5s
🟢	tests/wishlist.e2e.ts · 2 passed	1.6s
🟢	wishlist › saving toggles the button and the tab badge	269ms
🟢	wishlist › the wishlist tab renders the empty state despite the badge count (planted bug)	1.3s

_{e2e 0.15.2 · 1m 52s · web · run artifacts}

github-actions · 2026-10-02T10:33:23Z

🟢 e2e web agent: 32 passed

33 agent steps · 25 replayed from cache · 19 model calls · 71.7k tokens (92% cached) · $0.0033

All 32 tests in 5 files

	Test	Agent	Time
🟢	tests-agent/control-inventory.e2e.ts · 6 passed	6 steps	2.6s
🟢	control inventory › check sets the checkbox and the radio	1 step	531ms
🟢	control inventory › a long press and a double click are chosen over a plain tap	1 step	870ms
🟢	control inventory › a right-click opens the file menu and Rename renames the file	1 step	366ms
🟢	control inventory › upload attaches the named project files	1 step	243ms
🟢	control inventory › back returns from the details view to the inventory	1 step	245ms
🟢	control inventory › scroll_to brings the footnote into view	1 step	364ms
🟢	tests-agent/login-form.e2e.ts · 1 passed	1 step	337ms
🟢	act signs in with the benchmark account	1 step	337ms
🟢	tests-agent/planted-bugs.e2e.ts · 3 passed	4 steps · 7 calls	9.6s
🟢	cart totals: a judgment catches the stale order summary	2 steps · 3 calls	4.4s
🟢	checkout review: a judgment catches the broken copy	1 step · 1 call	1.3s
🟢	wishlist: a judgment catches the lost saved items	1 step · 3 calls	3.9s
🟢	tests-agent/scenarios.e2e.ts · 21 passed	21 steps · 12 calls	2m 33s
🟢	act completes shadow-dom-form	1 step · 3 calls	3.7s
🟢	act completes lying-labels	1 step · 3 calls	4.7s
🟢	act completes iframe-form	1 step · 3 calls	4.3s
🟢	act completes infinite-scroll	1 step	13.6s
🟢	act completes virtualized-table	1 step	1m 41s
🟢	act completes overlay-trap	1 step	823ms
🟢	act completes sticky-chrome	1 step	6.7s
🟢	act completes async-states	1 step · 3 calls	6.3s
🟢	act completes debounced-search	1 step	2.3s
🟢	act completes hover-menu	1 step	429ms
🟢	act completes drag-and-drop	1 step	420ms
🟢	act completes kanban-board	1 step	585ms
🟢	act completes context-menu-trap	1 step	428ms
🟢	act completes stale-dom	1 step	401ms
🟢	act completes otp-auto-advance	1 step	658ms
🟢	act completes date-picker	1 step	777ms
🟢	act completes rich-text-editor	1 step	1.3s
🟢	act completes playbook-cleanup	1 step	506ms
🟢	act completes promo-storefront	1 step	1.3s
🟢	act completes gift-card-purchase	1 step	2.4s
🟢	act filters the list and the URL carries the filter	1 step	179ms
🟢	tests-agent/unique-cache.e2e.ts · 1 passed	1 step	278ms
🟢	act adds an address with a run-unique label	1 step	278ms

_{e2e 0.15.2 · 2m 47s · web · run artifacts}

github-actions · 2026-10-02T10:39:25Z

🟢 e2e android: 54 passed, 1 skipped

All 55 tests in 7 files

	Test	Time
⏭️	tests/controls.e2e.ts · 24 passed, 1 skipped	2m 50s
🟢	control inventory › a label query resolves a labeled text field	23.7s
🟢	control inventory › a labeled field answers name, display value, value reads, fill, and clear	11.7s
🟢	control inventory › a placeholder query and attribute reads find the field	4.7s
🟢	control inventory › a secure field takes a fill and denies value reads	6.6s
🟢	control inventory › press sends a character and Enter through the soft keyboard	8.3s
🟢	control inventory › a field autofocuses on mount and focus() moves the focus	10.7s
🟢	control inventory › toBeFocused follows the focus	9.1s
🟢	control inventory › a switch checks, unchecks, and reports its state	10.1s
🟢	control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap	6.0s
🟢	control inventory › checkbox and radio roles check, uncheck, and report their state	11.2s
🟢	control inventory › a tab role answers getByRole	3.5s
🟢	control inventory › a radio group reports the checked option	8.3s
🟢	control inventory › buttons report disabled, enabled, and a state that arrives later	7.8s
🟢	control inventory › a display: none element is absent rather than hidden, its visible twin attached	5.7s
🟢	control inventory › a header answers its accessible name	3.4s
🟢	control inventory › a tab row answers count, all, texts, filters, positions, and switching	5.0s
🟢	control inventory › a list answers counts, list-form text, reads, and boxes in order	4.8s
⏭️	control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢	control inventory › a long press lands as one	6.8s
🟢	control inventory › a double tap lands inside the 300 ms window	7.6s
🟢	control inventory › an ambiguous locator fails at once	4.2s
🟢	control inventory › screenshot, restart, and clearState reopen the app on its home list	9.0s
🟢	refusals › verbs a device never declares are refused before the engine	3ms
🟢	refusals › keys the soft keyboard cannot send are refused by the engine	361ms
🟢	refusals › a missing locator fails as not found once its timeout passes	1.1s
🟢	tests/device-fixture.e2e.ts · 8 passed	1m 0s
🟢	device fixture › setOrientation rotates the window the app measures	4.4s
🟢	device fixture › home leaves the app, openApp brings it back where it was	5.3s
🟢	device fixture › foregroundApp sees the home screen after home()	3.9s
🟢	device fixture › closeApp then openApp relaunches on the home list	5.3s
🟢	device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN	3.6s
🟢	device fixture › device.back pops the scenario	4.3s
🟢	device fixture › setNetwork and setAirplaneMode change what the app sees	14.7s
🟢	device fixture › setLocation and clearLocation change what the app reads	18.7s
🟢	tests/device.e2e.ts · 4 passed	33.6s
🟢	permission prompt › allowing the system dialog enables the microphone	5.5s
🟢	permission prompt › a denied prompt recovers once the permission is granted outside	11.7s
🟢	sequential onboarding echoes the exact values	9.6s
🟢	onboarding rejects a malformed email before advancing	6.8s
🟢	tests/flows-lists.e2e.ts · 7 passed	2m 8s
🟢	infinite scroll reaches the target item	17.5s
🟢	product catalog adds exactly two of the right variant	10.1s
🟢	gestures: long-press, then swipe left	7.4s
🟢	gestures: double-tap completes the flow	9.0s
🟢	sticky chrome: accept below the fold, then continue	10.9s
🟢	async states: load, pull to refresh, claim through the toast	11.2s
🟢	huge virtualized list reaches row 512	1m 1s
🟢	tests/flows.e2e.ts · 7 passed	49.8s
🟢	text inputs unlock the submit button	6.0s
🟢	filled text inputs report their values	5.3s
🟢	modal flow confirms through the native alert	7.8s
🟢	bottom tabs act inside the last tab	6.8s
🟢	error recovery retries, then confirms the delete	8.0s
🟢	choice controls place the exact order	7.6s
🟢	debounced search selects the target once results arrive	8.2s
🟢	tests/login-form.e2e.ts · 3 passed	29.2s
🟢	login form › rejects a malformed email	8.3s
🟢	login form › rejects an unknown account	8.3s
🟢	login form › signs in and logs out	12.6s
🟢	tests/smoke.e2e.ts · 1 passed	5.9s
🟢	home lists the scenarios and opens one	5.9s

_{e2e 0.15.2 · 7m 56s · android-emulator · run artifacts}

github-actions · 2026-10-02T10:40:13Z

🟢 e2e android agent: 20 passed

24 agent steps · 110 model calls · 593.0k tokens (84% cached) · $0.0377

All 20 tests in 1 file

	Test	Agent	Time
🟢	tests-agent/scenarios.e2e.ts · 20 passed	24 steps · 110 calls	8m 41s
🟢	act completes Login Form	1 step · 4 calls	32.9s
🟢	act completes Infinite Scroll List	1 step · 3 calls	25.9s
🟢	act completes Modal Flow	1 step · 4 calls	11.6s
🟢	act completes Bottom Tabs	1 step · 3 calls	8.5s
🟢	act completes Text Input Variations	1 step · 3 calls	10.8s
🟢	act completes Broken Accessibility	2 steps · 6 calls	32.4s
🟢	act completes Vision Only	2 steps · 6 calls	28.1s
🟢	act completes Gestures	1 step · 10 calls	35.8s
🟢	act completes Async States	1 step · 8 calls	31.3s
🟢	act completes Debounced Search	1 step · 4 calls	10.5s
🟢	act completes Huge Virtualized List	1 step · 3 calls	1m 11s
🟢	act completes Flattened Registration Form	2 steps · 8 calls	46.4s
🟢	act completes Flattened Login	2 steps · 6 calls	26.7s
🟢	act completes Sticky Chrome Target	1 step · 4 calls	13.9s
🟢	act completes WebView Accessibility	1 step · 5 calls	14.4s
🟢	act completes Permission Prompt	1 step · 3 calls	8.6s
🟢	act completes Product Catalog	1 step · 5 calls	13.5s
🟢	act completes Error Recovery	1 step · 6 calls	13.3s
🟢	act completes Choice Controls	1 step · 7 calls	25.5s
🟢	act completes Sequential Onboarding	1 step · 12 calls	1m 0s

_{e2e 0.15.2 · 8m 42s · android-emulator · run artifacts}

github-actions · 2026-10-02T10:45:18Z

🟢 e2e ios: 1 flaky, 51 passed, 1 skipped

⚠️ 1 flaky test passed on a retry

⚠️ control inventory › toBeFocused follows the focus
tests/controls.e2e.ts:101

ASSERTION_FAILED at step 5 of 5: expect.toBeFocused getByRole("textbox", name: "Autofocus field"), after 5.1s

Expected: focused
Observed: default states (1 match)
Failed the same way on 2 attempts: ASSERTION_FAILED at step 5. The retry that passed is the exception.
Screen: dev.e2e.benchmark / Control Inventory

Evidence: screenshot, log

All 53 tests in 7 files

	Test	Time
⚠️	tests/controls.e2e.ts · 1 flaky, 22 passed, 1 skipped	5m 23s
🟢	control inventory › a label query resolves a labeled text field	35.7s
🟢	control inventory › a labeled field answers name, display value, value reads, fill, and clear	29.8s
🟢	control inventory › a placeholder query and attribute reads find the field	7.7s
🟢	control inventory › a secure field takes a fill and denies value reads	13.4s
🟢	control inventory › press sends a character and Enter through the soft keyboard	14.0s
🟢	control inventory › a field autofocuses on mount and focus() moves the focus	20.2s
⚠️	control inventory › toBeFocused follows the focus (2 failed attempts first)	14.4s
🟢	control inventory › a switch checks, unchecks, and reports its state	16.1s
🟢	control inventory › checkbox and radio roles answer getByRole, and a checkbox toggles by tap	13.3s
🟢	control inventory › checkbox and radio roles check, uncheck, and report their state	20.5s
🟢	control inventory › a radio group reports the checked option	14.7s
🟢	control inventory › buttons report disabled, enabled, and a state that arrives later	12.4s
🟢	control inventory › a display: none element is absent rather than hidden, its visible twin attached	12.5s
🟢	control inventory › a header answers its accessible name	8.3s
🟢	control inventory › a tab row answers count, all, texts, filters, positions, and switching	12.0s
🟢	control inventory › a list answers counts, list-form text, reads, and boxes in order	12.8s
⏭️	control inventory › a row contains its texts (skipped: React Native flattens a plain View out of the accessibility tree on both platforms: XCTest and the Android helper report a View with a testID as a leaf beside its children, so filter({ has }) and a query scoped to it match nothing; only Sc…)
🟢	control inventory › a long press lands as one	12.7s
🟢	control inventory › pointer verbs land at the point asked for	17.9s
🟢	control inventory › an ambiguous locator fails at once	9.9s
🟢	control inventory › screenshot, restart, and clearState reopen the app on its home list	22.4s
🟢	refusals › verbs a device never declares are refused before the engine	6ms
🟢	refusals › keys the soft keyboard cannot send are refused by the engine	633ms
🟢	refusals › a missing locator fails as not found once its timeout passes	1.2s
🟢	tests/device-fixture.e2e.ts · 9 passed	2m 33s
🟢	device fixture › setAppearance flips the color scheme the app reads	26.9s
🟢	device fixture › setOrientation rotates the window the app measures	11.4s
🟢	device fixture › the clipboard reads back what was written	10.0s
🟢	device fixture › home leaves the app, openApp brings it back where it was	10.6s
🟢	device fixture › closeApp then openApp relaunches on the home list	14.8s
🟢	device fixture › foregroundApp after closeApp reports the closed session as APP_NOT_OPEN	12.4s
🟢	device fixture › device.back pops the scenario	11.4s
🟢	device fixture › setLocation and clearLocation change what the app reads	28.1s
🟢	device fixture › enrollBiometrics and setBiometrics answer a biometric prompt	27.7s
🟢	tests/device.e2e.ts · 2 passed	1m 15s
🟢	sequential onboarding echoes the exact values	54.3s
🟢	onboarding rejects a malformed email before advancing	20.8s
🟢	tests/flows-lists.e2e.ts · 7 passed	5m 28s
🟢	infinite scroll reaches the target item	43.4s
🟢	product catalog adds exactly two of the right variant	15.9s
🟢	gestures: long-press, then swipe left	13.4s
🟢	sticky chrome: accept below the fold, then continue	17.8s
🟢	async states: load, pull to refresh, claim through the toast	17.0s
🟢	huge virtualized list reaches row 512	3m 25s
🟢	web view coupon form applies the code shown on the page	15.1s
🟢	tests/flows.e2e.ts · 7 passed	1m 48s
🟢	text inputs unlock the submit button	27.5s
🟢	filled text inputs report their values	16.5s
🟢	modal flow confirms through the native alert	12.1s
🟢	bottom tabs act inside the last tab	11.0s
🟢	error recovery retries, then confirms the delete	13.2s
🟢	choice controls place the exact order	12.4s
🟢	debounced search selects the target once results arrive	15.0s
🟢	tests/login-form.e2e.ts · 3 passed	1m 8s
🟢	login form › rejects a malformed email	16.7s
🟢	login form › rejects an unknown account	19.3s
🟢	login form › signs in and logs out	32.1s
🟢	tests/smoke.e2e.ts · 1 passed	16.3s
🟢	home lists the scenarios and opens one	16.3s

_{e2e 0.15.2 · 9m 36s · ios-simulator · run artifacts}

github-actions · 2026-10-02T10:53:44Z

🔴 e2e ios agent: 1 failed, 20 passed, 1 skipped

25 agent steps · 136 model calls · 148.9k tokens (82% cached) · $0.0112

🔴 act completes Gestures
tests-agent/scenarios.e2e.ts:257

ASSERTION_FAILED at step 4 of 4: agent.act "Complete this scenario as the screen instructs: long-press the first target, swipe the card left, then double-tap the last target", after 1m 4s and 12 model calls

agent.act failed: Long-pressed the first target and swiped the card left; both advanced the scenario. Double-tapped the last target twice, but the screen remained on Step 3 of 3 with “Double-tap the button below,” so completion was not confirmed.

Failed on all 4 attempts: ASSERTION_FAILED at step 4 (2 times), then ACTION_FAILED at step 5, then ASSERTION_FAILED at step 4.
Turn 9: double_tap({"target":"n684"}) → [double_tap] Double-tapped #n684.
Turn 10: tap_at({"x":177,"y":464}) → [tap_at] Tapped the point (202, 528); no listed control is there (under it: #n714 other "Double-tap me" testid="double-tap-target").
Turn 11: tap_at({"x":177,"y":464}) → [tap_at] Tapped the point (202, 528); no listed control is there (under it: #n744 other "Double-tap me" testid="double-tap-target").
Screen: dev.e2e.benchmark / Benchmark Examples

Evidence: screenshot, log

All 22 tests in 1 file

	Test	Agent	Time
🔴	tests-agent/scenarios.e2e.ts · 1 failed, 20 passed, 1 skipped	25 steps · 136 calls	15m 27s
🟢	act completes Login Form	1 step · 11 calls	1m 32s
🟢	act completes Infinite Scroll List	1 step · 3 calls	59.9s
🟢	act completes Modal Flow	1 step · 4 calls	28.1s
🟢	act completes Bottom Tabs	1 step · 3 calls	14.4s
🟢	act completes Text Input Variations	1 step · 3 calls	34.5s
🟢	act completes Broken Accessibility	2 steps · 7 calls	49.8s
🟢	act completes Vision Only	2 steps · 6 calls	42.3s
🔴	act completes Gestures	1 step · 12 calls	1m 11s
🟢	act completes Async States	1 step · 8 calls	43.6s
🟢	act completes Debounced Search	1 step · 4 calls	24.7s
🟢	act completes Flattened Registration Form	2 steps · 14 calls	1m 53s
🟢	act completes Flattened Login	2 steps · 12 calls	1m 38s
🟢	act completes Sticky Chrome Target	1 step · 7 calls	45.4s
🟢	act completes WebView Accessibility	1 step · 5 calls	42.0s
🟢	act completes Permission Prompt	1 step · 3 calls	23.3s
🟢	act completes Photo Picker	1 step · 4 calls	17.6s
🟢	act completes Product Catalog	1 step · 5 calls	16.0s
🟢	act completes Error Recovery	1 step · 6 calls	16.6s
🟢	act completes Choice Controls	1 step · 8 calls	39.3s
🟢	act completes Stripe PaymentSheet	1 step · 7 calls	36.6s
🟢	act completes Apple Pay	1 step · 4 calls	19.4s
⏭️	act completes Apple Pay Billing Address (skipped: the CI simulator's Apple Pay sheet skips the billing-address step, so the scenario cannot be completed there)

_{e2e 0.15.2 · 19m 38s · ios-simulator · run artifacts}

fix(oauth): list ChatGPT models as Codex CLI 0.160.0

31995c1

devin-ai-integration Bot requested review from okwasniewski and szymonrybczak as code owners October 2, 2026 10:29

devin-ai-integration Bot assigned okwasniewski Oct 2, 2026

okwasniewski merged commit b3c9832 into main Oct 2, 2026
14 of 15 checks passed

okwasniewski deleted the devin/1790936656-codex-client-version branch October 2, 2026 11:03

github-actions Bot mentioned this pull request Oct 2, 2026

chore: version packages #761

Merged

Navigation Menu

fix(oauth): list ChatGPT models as Codex CLI 0.160.0 - #760

fix(oauth): list ChatGPT models as Codex CLI 0.160.0#760
okwasniewski merged 1 commit into
mainfrom
devin/1790936656-codex-client-version

devin-ai-integration Bot commented Oct 2, 2026 •

edited by cubic-dev-ai Bot

Loading

Uh oh!

devin-ai-integration Bot commented Oct 2, 2026

Uh oh!

github-actions Bot commented Oct 2, 2026

Uh oh!

github-actions Bot commented Oct 2, 2026

Uh oh!

github-actions Bot commented Oct 2, 2026

Uh oh!

github-actions Bot commented Oct 2, 2026

Uh oh!

github-actions Bot commented Oct 2, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 2, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

devin-ai-integration Bot commented Oct 2, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What and why

Verified

Checklist

Uh oh!

devin-ai-integration Bot commented Oct 2, 2026

Uh oh!

github-actions Bot commented Oct 2, 2026

🟢 e2e web: 90 passed

Uh oh!

github-actions Bot commented Oct 2, 2026

🟢 e2e web agent: 32 passed

Uh oh!

github-actions Bot commented Oct 2, 2026

🟢 e2e android: 54 passed, 1 skipped

Uh oh!

github-actions Bot commented Oct 2, 2026

🟢 e2e android agent: 20 passed

Uh oh!

github-actions Bot commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 e2e ios: 1 flaky, 51 passed, 1 skipped

Uh oh!

github-actions Bot commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔴 e2e ios agent: 1 failed, 20 passed, 1 skipped

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

devin-ai-integration Bot commented Oct 2, 2026 •

edited by cubic-dev-ai Bot

Loading

github-actions Bot commented Oct 2, 2026 •

edited

Loading

github-actions Bot commented Oct 2, 2026 •

edited

Loading