Added robos.txt configuration instructions for CORTEX Configuration Portal and CORTEX Interaction Portal

content/en/docs/2026.3/getting-started/on-premise/add-configuration-portal-to-cortex/installation.md

@@ -228,6 +228,16 @@ On the Web Application Server:
 
 ### Configure the {{% ctx %}} Configuration Portal Website
 
+#### Copy robots.txt file
+
+To provide guidance for search engine crawlers and common AI bots not to index CORTEX Configuration Portal, it is necessary for the robots.txt file to reside in the site’s root directory of the CORTEX Configuration Portal. This is only necessary if CORTEX Configuration Portal is being installed on a different machine than CORTEX Gateway has been installed.
+
+1. Open the folder where `Cortex Innovation {{< version >}} - Configuration Portal.zip` was extracted to.
+1. Open the `Cortex Configuration Portal` folder.
+1. Copy the `robots.txt` file
+1. Navigate to the `Cortex` website directory, typically `C:\inetpub\wwwroot\Cortex`.
+1. Paste the `robots.txt` file
+
 #### Configure the Redirect rule
 
 On the Web Application Server:
@@ -239,6 +249,12 @@ On the Web Application Server:
     <add input="{REQUEST_URI}" pattern=".*\/ConfigurationPortal.*" negate="true" />
     ```
 
+1. If not present in the `Redirect Cortex to gateway` rule, add a condition as follows:
+
+    ``` xml
+    <add input="{REQUEST_URI}" pattern="^/robots\.txt$" negate="true" />
+    ```
+
 1. Save the file.
 
 #### CORS Configuration