Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
7f2770c
refactor: migrate act to conversation-based architecture and update c…
philipph-askui Feb 25, 2026
d79a5cd
feat: add caching_v2 features and fix otel dependency for tracing
philipph-askui Feb 25, 2026
835860a
feat: change default of `is_cacheable` flag to False
philipph-askui Feb 24, 2026
08a1a0e
fix: update prompts to state of caching_v02
philipph-askui Feb 25, 2026
ec8e82b
fix: format, typechecking, liniting issues
philipph-askui Feb 25, 2026
16983b4
fix: sanitizes messages before sending to API as we need to remove pr…
philipph-askui Feb 25, 2026
d6932f2
removes old 'llm_provider` field in CacheWritingSettings
philipph-askui Feb 25, 2026
297b5e3
fix: add default cache directory (.askui_cache) to gitignore
philipph-askui Feb 25, 2026
cc47181
chore: change logging outputs to INFO
philipph-askui Feb 25, 2026
6e3a52f
fix: removes old cache_writer and makes code use the new cache_manager
philipph-askui Feb 25, 2026
57b887d
fix: handles problems due to tools now having uuid suffixes
philipph-askui Feb 25, 2026
35d6149
fix: adds missing cache parameter handling
philipph-askui Feb 25, 2026
6f08bce
fix: update outdated tests
philipph-askui Feb 25, 2026
76daa92
feat: add method to truncate content for html reports to prevent floo…
philipph-askui Feb 26, 2026
16257dd
fix: migrate caching to conversation-based architecture and add missi…
philipph-askui Feb 26, 2026
a4c6449
fix: bug in visual validation during cached execution
philipph-askui Feb 26, 2026
ac2caf1
chore: change default value for `visual_validation_threshold` to 10
philipph-askui Feb 26, 2026
586aee3
fix: add explicit conversion to int of mouse move coordinats, as the …
philipph-askui Feb 26, 2026
aa93ff7
chore: change log message from warning to info
philipph-askui Feb 26, 2026
f04f1c9
fix: remove unnecessary files
philipph-askui Feb 26, 2026
908d55d
fix: duplicate clipping of coordinates
philipph-askui Feb 26, 2026
7f4b95f
fix: multiple bugs and code quality issues
philipph-askui Feb 26, 2026
0b9e13b
fix: change default value of `delay_time_between_actions` from 0.5 to…
philipph-askui Feb 26, 2026
b60d1bf
feat: add usage statistics of caching to html reporter
philipph-askui Feb 26, 2026
f99082f
fix: bug where cached executions were reported as success when they w…
philipph-askui Feb 26, 2026
fdece1d
fix: change name of caching strategies to match new pattern from cach…
philipph-askui Feb 26, 2026
6d961f8
fix: coding quality issue
philipph-askui Feb 27, 2026
1c266ae
chore: add pydantic model for VisualValidationMetadata
philipph-askui Feb 27, 2026
ee78cbf
chore: move conversation to models/shared
philipph-askui Feb 27, 2026
4029647
chore: refactor control loop and delete legacy code (custom_agent and…
philipph-askui Feb 27, 2026
d135838
feat: add callback system comparable to pytorch lightning
philipph-askui Feb 27, 2026
dfc9068
fix: bug in html report that led to a crash for non-cached executions
philipph-askui Mar 2, 2026
19ab09c
feat: add new speaker handoff pattern that is more scalable and gener…
philipph-askui Mar 2, 2026
9e3672b
feat: add conversation_id to conversation
philipph-askui Mar 2, 2026
908b5ce
feat: add on_speaker_switch callback
philipph-askui Mar 2, 2026
1650ff7
chore: resolve joint callback method into methods that handle them in…
philipph-askui Mar 2, 2026
6d17375
chore: refactor usage tracking to integrate via callback
philipph-askui Mar 2, 2026
f8b416a
chore: change name of caching strategy `both` to `auto`
philipph-askui Mar 3, 2026
33c72bf
Merge branch 'main' into chore/act_conversation_with_caching
philipph-askui Mar 3, 2026
0b1d45e
fix: add missing `_on_speaker_switch` callback to docs
philipph-askui Mar 4, 2026
efe0624
Merge branch 'chore/act_conversation_with_caching' of https://github.…
philipph-askui Mar 4, 2026
f3ca227
fix: make tracing span names consistent with function names
philipph-askui Mar 4, 2026
7d63869
chore: rename `handle_result_status` to `_handle_continue_conversation`
philipph-askui Mar 4, 2026
56d3793
chore: move speaker switch into a dedicated function `switch_speaker_…
philipph-askui Mar 4, 2026
6d9b7f4
chore: remove unused `_has_tool_calls` from AgentSpeaker
philipph-askui Mar 4, 2026
77239f0
Merge branch 'main' into chore/act_conversation_with_caching
philipph-askui Mar 4, 2026
70293a4
fix: linting issue (Line too long)
philipph-askui Mar 4, 2026
c5728db
chore: remove unused local `speaker` variable
philipph-askui Mar 4, 2026
3c8f50a
fix: run pdm install
philipph-askui Mar 4, 2026
1c1c687
chore: make description and name public member variables of speakers …
philipph-askui Mar 4, 2026
979ebba
chore: remove code quality (remove try-except, add isEnabledFor check…
philipph-askui Mar 4, 2026
ece5dfd
rename `_conclude_control_loop` to `_teardown_control_loop`
philipph-askui Mar 4, 2026
7998ac0
chore: refactor `_sanitize_message_for_api` into new `from_message_pa…
philipph-askui Mar 4, 2026
8d75ab6
fix: update docs to reflect latest changes
philipph-askui Mar 5, 2026
cb5f1a6
fix: remove try-except in CacheExecutor
philipph-askui Mar 5, 2026
62b817c
chore: add inline comment in ContentBlock conversion of anthropic mes…
philipph-askui Mar 5, 2026
7e04fb4
fix: exclude agent settings from telemetry as it cant be converted to…
philipph-askui Mar 5, 2026
0a286d3
fix: bug in agent response status
philipph-askui Mar 5, 2026
7d8c054
chore: set correct logger name
philipph-askui Mar 5, 2026
698596c
chore: clean up logging in cache verification
philipph-askui Mar 5, 2026
b6998bf
fix: refines cache use prompt to prevent the model from using the wro…
philipph-askui Mar 5, 2026
0b02f97
feat: adds hint to "Original" token values that this was the consumed…
philipph-askui Mar 5, 2026
b7d06af
chore: removes outdated `SIMPLIFICATION_CONCEPT.md`
philipph-askui Mar 5, 2026
0e42685
fix: index of docs in overivew to align with filenames
philipph-askui Mar 5, 2026
c85141f
feat: change default model for vlm_providers to `claude-sonnet-4-6`
philipph-askui Mar 5, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -169,5 +169,6 @@ reports/
/askui_chat.db-shm
/askui_chat.db-wal
.cache/
.askui_cache/*

bom.json
23 changes: 12 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,17 +112,18 @@ with ComputerAgent() as agent:

Ready to build your first agent? Check out our documentation:

1. **[Start Here](docs/00_overview.md)** - Overview and core concepts
2. **[Setup](docs/01_setup.md)** - Installation and configuration
3. **[Using Agents](docs/02_using_agents.md)** - Using the AskUI ComputerAgent and AndroidAgent
4. **[System Prompts](docs/03_prompting.md)** - How to write effective instructions
5. **[Using Models](docs/04_using_models.md)** - Using different models as backbone for act, get, and locate
6. **[BYOM](docs/05_bring_your_own_model_provider.md)** - use your own model cloud by plugging in your own model provider
7. **[Caching](docs/06_caching.md)** - Optimize performance and costs
8. **[Tools](docs/07_tools.md)** - Extend agent capabilities
9. **[Reporting](docs/08_reporting.md)** - Obtain agent logs as execution reports and summaries as test reports
10. **[Observability](docs/09_observability_telemetry_tracing.md)** - Monitor and debug agents
11. **[Extracting Data](docs/10_extracting_data.md)** - Extracting structured data from screenshots and files
0. **[Start Here](docs/00_overview.md)** - Overview and core concepts
1. **[Setup](docs/01_setup.md)** - Installation and configuration
2. **[Using Agents](docs/02_using_agents.md)** - Using the AskUI ComputerAgent and AndroidAgent
3. **[System Prompts](docs/03_prompting.md)** - How to write effective instructions
4. **[Using Models](docs/04_using_models.md)** - Using different models as backbone for act, get, and locate
5. **[BYOM](docs/05_bring_your_own_model_provider.md)** - use your own model cloud by plugging in your own model provider
6. **[Caching](docs/06_caching.md)** - Optimize performance and costs
7. **[Tools](docs/07_tools.md)** - Extend agent capabilities
8. **[Reporting](docs/08_reporting.md)** - Obtain agent logs as execution reports and summaries as test reports
9. **[Observability](docs/09_observability_telemetry_tracing.md)** - Monitor and debug agents
10. **[Extracting Data](docs/10_extracting_data.md)** - Extracting structured data from screenshots and files
11. **[Callbacks](docs/11_callbacks.md)** - Inject custom logic into the control loop

**Official documentation:** [docs.askui.com](https://docs.askui.com)

Expand Down
138 changes: 0 additions & 138 deletions SIMPLIFICATION_CONCEPT.md

This file was deleted.

3 changes: 3 additions & 0 deletions docs/00_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,9 @@ Understand what data is collected and how to opt out.
### 10 - Extracting Data
**Topics**: Using `get()`, file support (PDF. Excel, Word, CSV), structured data extraction, response schemas

### 11 - Callbacks
**Topics**: Inject custom logic at different positions of the control loop through callbacks

Extract information from screens and files using the `get()` method with Pydantic models.

## Additional Resources
Expand Down
6 changes: 3 additions & 3 deletions docs/03_prompting.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ with WebVisionAgent() as agent:
"Log in with username 'testuser' and password 'testpass123'",
# CAUTION: this will also override all other MessageSettings
# eventually provided earlier!
settings=ActSettings(messages=MessageSettings(system=prompt))
act_settings=ActSettings(messages=MessageSettings(system=prompt))
)
```

Expand Down Expand Up @@ -170,7 +170,7 @@ prompt = ActSystemPrompt(
with WebVisionAgent() as agent:
agent.act(
"Your task",
settings=ActSettings(messages=MessageSettings(system=prompt))
act_settings=ActSettings(messages=MessageSettings(system=prompt))
)
```

Expand Down Expand Up @@ -329,6 +329,6 @@ with WebVisionAgent() as agent:
"Find a laptop under $1000 and add it to cart",
# CAUTION: this will also override all other MessageSettings
# eventually provided earlier!
settings=ActSettings(messages=MessageSettings(system=prompt))
act_settings=ActSettings(messages=MessageSettings(system=prompt))
)
```
7 changes: 3 additions & 4 deletions docs/05_bring_your_own_model_provider.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ Environment variables:

If you want to use Anthropic models directly from the Anthropic API with your Anthropic API key, you can use:
- `AnthropicVlmProvider` for `act()`
- `AntrhopicImageQAProvider` for `get()`
- `AnthropicImageQAProvider` for `get()`

```python
import os
Expand Down Expand Up @@ -78,12 +78,11 @@ If you want to use Google models directly from the Google API with your Google A
- `GoogleImageQAProvider` for `get()`

```python
import os
from askui import AgentSettings, ComputerAgent
from askui.model_providers import AnthropicVlmProvider
from askui.model_providers import GoogleImageQAProvider

with ComputerAgent(settings=AgentSettings(
vlm_provider=GoogleImageQAProvider(
image_qa_provider=GoogleImageQAProvider(
model_id="gemini-2.5-pro",
),
)) as agent:
Expand Down
Loading