Skip to main content

Observe

See how to use observe() to discover actionable elements and analyze web page structure

​
Method Signatures

  • TypeScript
  • Python
// With ObserveOptions
await page.observe(options: ObserveOptions): Promise<ObserveResult[]>

// String instruction shorthand
await page.observe(instruction: string): Promise<ObserveResult[]>
ObserveOptions Interface:
interface ObserveOptions {
  instruction?: string;
  modelName?: AvailableModel;
  modelClientOptions?: ClientOptions;
  domSettleTimeoutMs?: number;
  drawOverlay?: boolean;
  iframes?: boolean;
}

​
Parameters

​
instruction
string
Natural language description of elements or actions to discover.
​
modelName
AvailableModel
Override the default LLM model for this observation.
​
modelClientOptions
ClientOptions
Model-specific configuration options.
​
domSettleTimeoutMs
number
Maximum time to wait for DOM to stabilize before analysis.Default: 30000
​
drawOverlay
boolean
Whether to draw visual overlays on discovered elements (debugging).Default: false
​
iframes
boolean
Set to true to search within iframes.Default: false

​
Response

Returns: Promise<ObserveResult[]> Array of discovered actionable elements, ordered by relevance. ObserveResult Interface: 
interface ObserveResult {
  selector: string;        // XPath selector to locate element
  description: string;     // Human-readable description
  method?: string;         // Suggested action method
  arguments?: string[];    // Additional action parameters
}
​
selector
string
XPath selector that precisely locates the element.
​
description
string
Human-readable description of the element and its purpose.
​
method
string
Suggested interaction method: "click", "fill", "selectOptionFromDropdown", "nextChunk", "scrollTo", "prevChunk".
​
arguments
string[]
Additional parameters for the suggested action.

​
Code Examples

// Basic element discovery
const buttons = await page.observe("find all clickable buttons");
const formFields = await page.observe("locate form input fields");

// Advanced configuration
const elements = await page.observe({
  instruction: "find important call-to-action buttons",
  modelName: "gpt-4.1-mini",
  domSettleTimeoutMs: 45000,
  drawOverlay: true
});

// Working with results
const [loginButton] = await page.observe("find the login button");
if (loginButton) {
  console.log("Found:", loginButton.description);
  console.log("Selector:", loginButton.selector);
  await page.act(loginButton); // Execute the action
}

// Filter results
const submitButtons = await page.observe("find all submit buttons");
const primarySubmit = submitButtons.find(btn => 
  btn.description.toLowerCase().includes('primary')
);

// Iframe search
const iframeElements = await page.observe({
  instruction: "find form fields inside the iframe",
  iframes: true
});

​
Integration Patterns

// Observe → Act workflow
const actions = await page.observe("find checkout elements");
for (const action of actions) {
  await page.act(action);
  await page.waitForTimeout(1000);
}

// Observe → Extract workflow
const tables = await page.observe("find data tables");
if (tables.length > 0) {
  const data = await page.extract({
    instruction: "extract the table data",
    selector: tables[0].selector,
    schema: DataSchema
  });
}

// Element validation
const requiredElements = await page.observe("find the login form");
if (requiredElements.length === 0) {
  throw new Error("Login form not found");
}
⌘I