Implement Bitrix24Partners support (issue #70)#109
Open
KarlsonComplete wants to merge 58 commits into
Conversation
- Replace `docker-compose` with `docker compose` commands in `Makefile` - Remove deprecated `setBitrix24PartnerId` method from `Bitrix24Partner` class
…tory` instance and simplify `setPhone` logic in `Bitrix24Partner`.
…improved context.
…andler создания партнера.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors the Bitrix24Partners module by renaming bitrix24PartnerId to bitrix24PartnerNumber and transitioning the repository to a composition-based approach. It also introduces email validation, updates PHP and PHPUnit version requirements, adds a maintainer skill definition, and improves testing infrastructure with a new partner builder. Feedback identifies inconsistent Docker command usage in the Makefile, suggests a more standard PHP version constraint, notes PSR-12 style regressions, recommends better HTML error handling, and identifies a typo in a directory name.
camaxtly
requested changes
Apr 19, 2026
…cked, MarkAsActive и Update.
…logic: - Add `Bitrix24PartnerNotFoundException` handling to UseCase Handlers (`Create`, `Update`, `Delete`, `MarkAsBlocked`, `MarkAsActive`) for better error management and logging. - Enhance field validation in `Update.Command` to ensure stricter checks. - Introduce `logoUrl` field in `Bitrix24Partner` entity and adjust related UseCases and tests. - Refactor tests to include new scenarios and improve data consistency.
- Replace `docker-compose` with `docker compose` in `Makefile`. - Introduce `bitrix24:partners:scrape:v2` command with updated logic. - Register `ScrapePartnersCommand_V2` in the console application.
- Integrate Monolog for detailed logging throughout the scraping process. - Refactor `ScrapePartnersCommand_V2` to enhance parsing logic and improve error handling. - Introduce support for fetching partner logos and storing results in a CSV file. - Streamline HTTP requests and implement configurable delays.
…`Makefile` and `console` setup.
…а, изменил сигнатуры методов сущности и добавил поддержку логотипа партнера.
KarlsonComplete
changed the title
camaxtly
reviewed
Apr 26, 2026
camaxtly
reviewed
Apr 26, 2026
camaxtly
requested changes
Apr 26, 2026
- Add `Uuid::v7()` support in `Bitrix24Partner` constructors and builders. - Improve validation for partner creation with `bitrix24PartnerNumber`. - Update Doctrine mapping for `bitrix24PartnerNumber` to use `unique` attribute.
…erBuilder: - Remove redundant `withBitrix24PartnerNumber` calls from tests. - Adjust `Bitrix24PartnerBuilder` to expand possible number range. - Reorder `clear()` in `Update.HandlerTest` for better test consistency.
…dize database truncation across functional tests.
- Implement stricter non-empty string checks for `site`, `email`, `openLineId`, `externalId`, and `logoUrl` in `Update.Command` and `Bitrix24Partner` entity. - Update associated tests to cover new validation scenarios.
…s, and refactor `Makefile` and `console` setup.
…aper` services and integrated them into `ScrapePartnersCommandV2` for improved modularity and reusability.
…ndling with new helper methods.
…ScrapePartnersCommandV2: добавлена обработка блокировок, настройки задержек, детектирование пустых страниц и обновлён механизм работы с доменом.
…rtnersCsvCommand, улучшил обработку пустых страниц и блокировок в ScrapePartnersCommandV2, добавил загрузку .env.local.
- Implement `Command` and `Handler` classes for upsert functionality. - Add validation for required fields in `Upsert\Command`. - Introduce `Upsert\Handler` to handle create, update, or skip logic based on partner data. - Update documentation to include new upsert scenarios and sync modes (`full`, `partial`). - Add unit and functional tests to cover all upsert scenarios, including creation, update, and skip cases. - Refactor partner import logic to align with the new upsert behavior.
…arser dependency: - Move PartnerHtmlParser logic into PartnerPageScraper as a dependency. - Simplify scraper command implementations (`UpdatePartnersCommand`, `ScrapePartnersCommand`) by using the consolidated `fetchPartnerData` method. - Introduce verbosity-based output control in `ScrapePartnersCommand`. - Add `PartnerData` DTO to structure scraped partner data. - Update tests to align with new scraping logic and enhance maintainability.
…ity: - Introduce `ScrapeWorkflow` to centralize scraping flow and state management. - Add contextual verbosity-based output handling to `ScrapePartnersCommand` and `UpdatePartnersCommand`. - Replace raw data arrays with `ScrapeResult` and `PartnerData` DTOs for better type safety. - Streamline `ScrapeStateManager` to improve progress tracking and state resumption. - Refactor utility methods across `PartnerPageScraper` for better extensibility and logging.
…bility: - Move scraper-related DTOs (`PartnerData`, `ScrapeResult`, `ScrapeConfig`) to `UseCase\Scrape` namespace. - Replace `ScrapeWorkflow` and `UpdateWorkflow` in `Infrastructure` with modular workflows under `UseCase\Scrape`. - Refactor `UpdatePartnersCommand` to use `UpdateWorkflow` and streamline options. - Improve type safety and replace raw arrays with DTOs across scraping functionality. - Migrate tests to match namespace changes and cover new workflows. - Remove deprecated scraper classes (`ScrapeWorkflow`, legacy DTOs).
- Standardize variable naming (`$e` to `$throwable`) for consistency. - Simplify conditional branches in `ScrapePartnersCommand`. - Add minor formatting adjustments (empty lines) for better code clarity.
camaxtly
reviewed
May 17, 2026
…raper commands refactoring.
…gional configurations: - Introduce `BanDetector` to identify scraping bans based on empty page patterns. - Replace `base-domain` inputs with `zone` (`ru`, `kz`) to centralize region handling. - Refactor workflows (`ScrapeWorkflow`, `UpdateWorkflow`) to integrate ban detection and handle new `zone` parameter. - Adjust commands (`ScrapePartnersCommand`, `UpdatePartnersCommand`) for streamlined regional scraping and improved verbosity. - Update DTOs (`ScrapeConfig`, `PartnerData`, `ScrapeResult`) to incorporate zone-specific logic and skipped partner tracking. - Migrate CSV file format to include `zone` instead of `base_domain`. - Add tests and documentation updates for new functionality and input changes.
…Bitrix24 PHP library maintainer skill files.
…apeWorkflow` for simplified CSV handling. Update `Bitrix24Zone` base domain URLs.
…ity, and improve progress reporting. Update Bitrix24 zone handling for domain and URL resolution.
…Command`: `partner-delay` и `page-delay` заменены на `partner-detail-delay` и `catalog-page-delay` соответственно, добавил корректную обработку заголовков карточек партнёров в парсере и обновил документацию о новой структуре опций.
…вил документацию по командам скрейпинга и импорта партнёров.
…ацию всех пунктов, включая BanDetector, processPartner, Bitrix24Zone enum, и обновление документации.
camaxtly
requested changes
May 24, 2026
…r update logic into `ScrapeWorkflow` and `ScrapePartnersCommand` for improved modularity and maintainability. Update ScrapeConfig to support specific partner updates. Streamline progress tracking and error handling.
…r file organization: - Replace `outputFile` with `outputDir` across configurations, workflows, and commands. - Introduce timestamped filenames for output files. - Update state management to support directory-based operations. - Adjust tests, Makefile commands, and verbosity outputs accordingly.
…scrape configuration to introduce `requestDelay`. Simplify `ScrapeWorkflow` start context resolution and improve state management consistency.
- Remove `.opencode/plans/2026-05-26-fix-output-dir-resume-logic.md` to declutter repository. - Refactor `ScrapeConfig` to use nullable `outputFile` and remove auto-generation. - Update `ScrapeStateManager` to store full `output_file` paths and add output-dir validation on resume. - Split `resolveStartContext` into `resolveResumeContext` and `getPageRange` in `ScrapeWorkflow`. - Refactor `ScrapePartnersCommand` for single-pass output-file resolution and verbosity improvements. - Introduce `ScrapeOptions` for streamlined scraping configurations.
…ortStats`, update method signatures, and streamline CSV file handling and import command default directory.
…leted partners, add option to include deleted in queries, and improve logging for skipped operations.
…enarios in `partner-scenarios.md` - Introduce `partner-common.md` with guidelines for CSV format, ban detection, and partner handling. - Simplify and consolidate `partner-scenarios.md` to focus on key scraping use cases. - Replace per-scenario options with centralized documentation for `partners:scrape` command. - Update examples and workflows to reflect streamlined scraping architecture and functionality.
…emove obsolete `Upsert` tests, commands, and handlers; add `Import` equivalents.
…ated use cases; streamline `Command` test cases by removing redundant assertions.
… maintainability: - Replace redundant `(new Bitrix24PartnerBuilder())` with `new Bitrix24PartnerBuilder()` across test files for cleaner syntax. - Remove unused properties and refine logic in `ScrapeProgress`. - Introduce `phpstan-baseline.neon` for static analysis error management. - Update `ScrapeConfig` and commands to align with stricter type declarations and nullable handling. - Bump `phpstan` and `rector` versions in `composer.json` for enhanced static analysis. - Restructure and streamline test scenarios for `MarkAsActive`, `Update`, `Delete`, and `Import`.
…ameter in constructor calls.
…age, commands, options, workflows, and error handling.
camaxtly
requested changes
May 31, 2026
|
|
||
| ## Партнёры без детальной страницы | ||
|
|
||
| Если детальная страница партнёра недоступна (нет ссылки на карточку), он пропускается — не записывается в CSV. ID пропущенных партнёров выводятся в лог и в финальный отчёт. Их можно позже обновить через `partners:scrape --partner-ids=...` (см. [partner-scenarios.md](partner-scenarios.md)). |
Collaborator
There was a problem hiding this comment.
тут только общее, если нужны детали дай ссылку на доку
| - **Ключ совпадения:** `bitrix24_partner_number` | ||
| - Команда: `bitrix24:partners:import` | ||
| - CSV-формат см. в [partner-common.md](partner-common.md) | ||
| - При ошибке в строке CSV импорт останавливается (fail-fast). Для проверки используйте `--dry-run` |
Collaborator
There was a problem hiding this comment.
Если нужен дру-ран то опиши его отдельно. Мы сами в csv пишем, ошибок с записью быть не должно.
|
|
||
| | Опция | Значения | По умолчанию | Описание | | ||
| |-------|----------|--------------|----------| | ||
| | `file` (аргумент) | путь | — | Путь к CSV файлу (обязательный). Если без пути — ищется в `var/scraper/` | |
Collaborator
There was a problem hiding this comment.
Сделай просто его обязательным и все. Меньше альтернатив, меньше ошибок.
| CSV = полный источник истины. Действия: **создание**, **обновление**, **пропуск**, **софт-делит**. | ||
|
|
||
| ```bash | ||
| php bin/console bitrix24:partners:import var/scraper/partners.csv |
Collaborator
There was a problem hiding this comment.
путь указываешь абсолютный, лишбо явно говоришь, что он относительный
|
|
||
| --- | ||
|
|
||
| ## Отчёт после выполнения |
Collaborator
There was a problem hiding this comment.
Отчет пользователь итак увидит, зачем его в доке писать? Имхо стоит удалить
| /** | ||
| * @return array<Bitrix24PartnerInterface> | ||
| */ | ||
| public function findAllActive(): array |
Collaborator
There was a problem hiding this comment.
или ты генератор возвращаешь, или страницу.
| withDeleted: true | ||
| ); | ||
|
|
||
| if (null !== $command->phone) { |
Collaborator
There was a problem hiding this comment.
Логическое противоречие, партнеров без телефона ты записываешь, но если у партнера телефон не корректный, то ты его пропускаешь.
| return; | ||
| } | ||
|
|
||
| if (!$existingPartner instanceof Bitrix24Partner) { |
Collaborator
There was a problem hiding this comment.
код не очень нужен, если реализация, то работаешь с реализацией. Если интерфейс, то интерфейс.
| ]); | ||
| } | ||
|
|
||
| private function phonesEqual(?PhoneNumber $a, ?PhoneNumber $b): bool |
Collaborator
There was a problem hiding this comment.
переписать с возможностями php8+
| use League\Csv\Writer; | ||
| use Psr\Log\LoggerInterface; | ||
|
|
||
| class ScrapeWorkflow |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This commit implements comprehensive support for Bitrix24Partners following the established DDD/CQRS patterns in the codebase.
Features Implemented:
Core Domain
Use Cases (CQRS)
CLI Tools
Testing
Technical Details:
Resolves: #70