Merge branch 'main' into support-halffloat-arrays/146238

Author: vstinner

Doc/library/getpass.rst

@@ -39,13 +39,27 @@ The :mod:`!getpass` module provides two functions:
       On Unix systems, when *echo_char* is set, the terminal will be
       configured to operate in
       :manpage:`noncanonical mode <termios(3)#Canonical_and_noncanonical_mode>`.
-      In particular, this means that line editing shortcuts such as
-      :kbd:`Ctrl+U` will not work and may insert unexpected characters into
-      the input.
+      Common terminal control characters are supported:
+
+      * :kbd:`Ctrl+A` - Move cursor to beginning of line
+      * :kbd:`Ctrl+E` - Move cursor to end of line
+      * :kbd:`Ctrl+K` - Kill (delete) from cursor to end of line
+      * :kbd:`Ctrl+U` - Kill (delete) entire line
+      * :kbd:`Ctrl+W` - Erase previous word
+      * :kbd:`Ctrl+V` - Insert next character literally (quote)
+      * :kbd:`Backspace`/:kbd:`DEL` - Delete character before cursor
+
+      These shortcuts work by reading the terminal's configured control
+      character mappings from termios settings.
 
    .. versionchanged:: 3.14
       Added the *echo_char* parameter for keyboard feedback.
 
+   .. versionchanged:: next
+      When using non-empty *echo_char* on Unix, keyboard shortcuts (including
+      cursor movement and line editing) are now properly handled using the
+      terminal's control character configuration.
+
 .. exception:: GetPassWarning
 
    A :exc:`UserWarning` subclass issued when password input may be echoed.

Doc/library/profiling.sampling.rst

@@ -1003,6 +1003,47 @@ at the top indicate functions that consume significant time either directly
 or through their callees.
 
 
+Differential flame graphs
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Differential flame graphs compare two profiling runs to highlight where
+performance changed. This helps identify regressions introduced by code changes
+and validate that optimizations achieved their intended effect::
+
+   # Capture baseline profile
+   python -m profiling.sampling run --binary -o baseline.bin script.py
+
+   # After modifying code, generate differential flamegraph
+   python -m profiling.sampling run --diff-flamegraph baseline.bin -o diff.html script.py
+
+The visualization draws the current profile with frame widths showing current
+time consumption, then applies color to indicate how each function changed
+relative to the baseline.
+
+**Color coding**:
+
+- **Red**: Functions consuming more time (regressions). Lighter shades indicate
+  modest increases, while darker shades show severe regressions.
+
+- **Blue**: Functions consuming less time (improvements). Lighter shades for
+  modest reductions, darker shades for significant speedups.
+
+- **Gray**: Minimal or no change.
+
+- **Purple**: New functions not present in the baseline.
+
+Frame colors indicate changes in **direct time** (time when the function was at
+the top of the stack, actively executing), not cumulative time including callees.
+Hovering over a frame shows comparison details including baseline time, current
+time, and the percentage change.
+
+Some call paths may disappear entirely between profiles. These are called
+**elided stacks** and occur when optimizations eliminate code paths or certain
+branches stop executing. If elided stacks are present, an elided toggle appears
+allowing you to switch between the main differential view and an elided-only
+view that shows just the removed paths (colored purple).
+
+
 Gecko format
 ------------
 
@@ -1488,6 +1529,10 @@ Output options
 
    Generate self-contained HTML flame graph.
 
+.. option:: --diff-flamegraph <baseline.bin>
+
+   Generate differential flamegraph comparing to a baseline binary profile.
+
 .. option:: --gecko
 
    Generate Gecko JSON format for Firefox Profiler.

Doc/library/select.rst

@@ -62,7 +62,7 @@ The module defines the following:
 
    *sizehint* informs epoll about the expected number of events to be
    registered.  It must be positive, or ``-1`` to use the default. It is only
-   used on older systems where :c:func:`!epoll_create1` is not available;
+   used on older systems where :manpage:`epoll_create1(2)` is not available;
    otherwise it has no effect (though its value is still checked).
 
    *flags* is deprecated and completely ignored.  However, when supplied, its
@@ -89,6 +89,11 @@ The module defines the following:
       The *flags* parameter.  ``select.EPOLL_CLOEXEC`` is used by default now.
       Use :func:`os.set_inheritable` to make the file descriptor inheritable.
 
+   .. versionchanged:: next
+
+      When CPython is built, this function may be disabled using
+      :option:`--disable-epoll`.
+
 
 .. function:: poll()
 

Doc/library/socket.rst

@@ -39,6 +39,8 @@ is implicit on send operations.
       A TLS/SSL wrapper for socket objects.
 
 
+.. _socket-addresses:
+
 Socket families
 ---------------
 
@@ -903,7 +905,7 @@ The following functions all create :ref:`socket objects <socket-objects>`.
 
    Build a pair of connected socket objects using the given address family, socket
    type, and protocol number.  Address family, socket type, and protocol number are
-   as for the :func:`~socket.socket` function above. The default family is :const:`AF_UNIX`
+   as for the :func:`~socket.socket` function. The default family is :const:`AF_UNIX`
    if defined on the platform; otherwise, the default is :const:`AF_INET`.
 
    The newly created sockets are :ref:`non-inheritable <fd_inheritance>`.
@@ -999,8 +1001,8 @@ The following functions all create :ref:`socket objects <socket-objects>`.
 
    Duplicate the file descriptor *fd* (an integer as returned by a file object's
    :meth:`~io.IOBase.fileno` method) and build a socket object from the result.  Address
-   family, socket type and protocol number are as for the :func:`~socket.socket` function
-   above. The file descriptor should refer to a socket, but this is not checked ---
+   family, socket type and protocol number are as for the :func:`~socket.socket` function.
+   The file descriptor should refer to a socket, but this is not checked ---
    subsequent operations on the object may fail if the file descriptor is invalid.
    This function is rarely needed, but can be used to get or set socket options on
    a socket passed to a program as standard input or output (such as a server
@@ -1564,8 +1566,8 @@ to sockets.
 
 .. method:: socket.bind(address)
 
-   Bind the socket to *address*.  The socket must not already be bound. (The format
-   of *address* depends on the address family --- see above.)
+   Bind the socket to *address*.  The socket must not already be bound. The format
+   of *address* depends on the address family --- see :ref:`socket-addresses`.
 
    .. audit-event:: socket.bind self,address socket.socket.bind
 
@@ -1598,8 +1600,8 @@ to sockets.
 
 .. method:: socket.connect(address)
 
-   Connect to a remote socket at *address*. (The format of *address* depends on the
-   address family --- see above.)
+   Connect to a remote socket at *address*. The format of *address* depends on the
+   address family --- see :ref:`socket-addresses`.
 
    If the connection is interrupted by a signal, the method waits until the
    connection completes, or raises a :exc:`TimeoutError` on timeout, if the
@@ -1674,16 +1676,16 @@ to sockets.
 .. method:: socket.getpeername()
 
    Return the remote address to which the socket is connected.  This is useful to
-   find out the port number of a remote IPv4/v6 socket, for instance. (The format
-   of the address returned depends on the address family --- see above.)  On some
-   systems this function is not supported.
+   find out the port number of a remote IPv4/v6 socket, for instance. The format
+   of the address returned depends on the address family --- see :ref:`socket-addresses`.
+   On some systems this function is not supported.
 
 
 .. method:: socket.getsockname()
 
    Return the socket's own address.  This is useful to find out the port number of
-   an IPv4/v6 socket, for instance. (The format of the address returned depends on
-   the address family --- see above.)
+   an IPv4/v6 socket, for instance. The format of the address returned depends on
+   the address family --- see :ref:`socket-addresses`.
 
 
 .. method:: socket.getsockopt(level, optname[, buflen])
@@ -1795,7 +1797,8 @@ to sockets.
    where *bytes* is a bytes object representing the data received and *address* is the
    address of the socket sending the data.  See the Unix manual page
    :manpage:`recv(2)` for the meaning of the optional argument *flags*; it defaults
-   to zero. (The format of *address* depends on the address family --- see above.)
+   to zero. The format of *address* depends on the address family --- see
+   :ref:`socket-addresses`.
 
    .. versionchanged:: 3.5
       If the system call is interrupted and the signal handler does not raise
@@ -1925,8 +1928,8 @@ to sockets.
    new bytestring.  The return value is a pair ``(nbytes, address)`` where *nbytes* is
    the number of bytes received and *address* is the address of the socket sending
    the data.  See the Unix manual page :manpage:`recv(2)` for the meaning of the
-   optional argument *flags*; it defaults to zero.  (The format of *address*
-   depends on the address family --- see above.)
+   optional argument *flags*; it defaults to zero. The format of *address*
+   depends on the address family --- see :ref:`socket-addresses`.
 
 
 .. method:: socket.recv_into(buffer[, nbytes[, flags]])
@@ -1941,7 +1944,7 @@ to sockets.
 .. method:: socket.send(bytes[, flags])
 
    Send data to the socket.  The socket must be connected to a remote socket.  The
-   optional *flags* argument has the same meaning as for :meth:`recv` above.
+   optional *flags* argument has the same meaning as for :meth:`recv`.
    Returns the number of bytes sent. Applications are responsible for checking that
    all data has been sent; if only some of the data was transmitted, the
    application needs to attempt delivery of the remaining data. For further
@@ -1956,7 +1959,7 @@ to sockets.
 .. method:: socket.sendall(bytes[, flags])
 
    Send data to the socket.  The socket must be connected to a remote socket.  The
-   optional *flags* argument has the same meaning as for :meth:`recv` above.
+   optional *flags* argument has the same meaning as for :meth:`recv`.
    Unlike :meth:`send`, this method continues to send data from *bytes* until
    either all data has been sent or an error occurs.  ``None`` is returned on
    success.  On error, an exception is raised, and there is no way to determine how
@@ -1977,9 +1980,9 @@ to sockets.
 
    Send data to the socket.  The socket should not be connected to a remote socket,
    since the destination socket is specified by *address*.  The optional *flags*
-   argument has the same meaning as for :meth:`recv` above.  Return the number of
-   bytes sent. (The format of *address* depends on the address family --- see
-   above.)
+   argument has the same meaning as for :meth:`recv`.  Return the number of
+   bytes sent. The format of *address* depends on the address family --- see
+   :ref:`socket-addresses`.
 
    .. audit-event:: socket.sendto self,address socket.socket.sendto
 

Doc/using/configure.rst

@@ -463,6 +463,17 @@ General Options
 
    ``pkg-config`` options.
 
+.. option:: --disable-epoll
+
+   Build without ``epoll``, meaning that :py:func:`select.epoll` will not be
+   present even if the system provides an
+   :manpage:`epoll_create <epoll_create(2)>` function.
+   This may be used on systems where :manpage:`!epoll_create` or
+   :manpage:`epoll_create1 <epoll_create1(2)>` is available
+   but incompatible with Linux semantics.
+
+   .. versionadded:: next
+
 
 C compiler options
 ------------------

Include/internal/pycore_ceval.h

@@ -302,7 +302,7 @@ PyAPI_FUNC(int) _PyEval_ExceptionGroupMatch(_PyInterpreterFrame *, PyObject* exc
 PyAPI_FUNC(void) _PyEval_FormatAwaitableError(PyThreadState *tstate, PyTypeObject *type, int oparg);
 PyAPI_FUNC(void) _PyEval_FormatExcCheckArg(PyThreadState *tstate, PyObject *exc, const char *format_str, PyObject *obj);
 PyAPI_FUNC(void) _PyEval_FormatExcUnbound(PyThreadState *tstate, PyCodeObject *co, int oparg);
-PyAPI_FUNC(void) _PyEval_FormatKwargsError(PyThreadState *tstate, PyObject *func, PyObject *kwargs);
+PyAPI_FUNC(void) _PyEval_FormatKwargsError(PyThreadState *tstate, PyObject *func, PyObject *kwargs, PyObject *dupkey);
 PyAPI_FUNC(PyObject *) _PyEval_ImportFrom(PyThreadState *, PyObject *, PyObject *);
 
 PyAPI_FUNC(PyObject *) _PyEval_LazyImportName(

Include/internal/pycore_debug_offsets.h

@@ -222,6 +222,8 @@ typedef struct _Py_DebugOffsets {
         uint64_t size;
         uint64_t collecting;
         uint64_t frame;
+        uint64_t generation_stats_size;
+        uint64_t generation_stats;
     } gc;
 
     // Generator object offset;
@@ -373,6 +375,8 @@ typedef struct _Py_DebugOffsets {
         .size = sizeof(struct _gc_runtime_state), \
         .collecting = offsetof(struct _gc_runtime_state, collecting), \
         .frame = offsetof(struct _gc_runtime_state, frame), \
+        .generation_stats_size = sizeof(struct gc_stats), \
+        .generation_stats = offsetof(struct _gc_runtime_state, generation_stats), \
     }, \
     .gen_object = { \
         .size = sizeof(PyGenObject), \

Include/internal/pycore_dict.h

@@ -55,7 +55,7 @@ extern Py_ssize_t _PyDict_SizeOf_LockHeld(PyDictObject *);
    of a key wins, if override is 2, a KeyError with conflicting key as
    argument is raised.
 */
-PyAPI_FUNC(int) _PyDict_MergeEx(PyObject *mp, PyObject *other, int override);
+PyAPI_FUNC(int) _PyDict_MergeUniq(PyObject *mp, PyObject *other, PyObject **dupkey);
 
 extern void _PyDict_DebugMallocStats(FILE *out);
 

Include/internal/pycore_interp_structs.h

@@ -177,31 +177,54 @@ struct gc_generation {
                   generations */
 };
 
-struct gc_collection_stats {
-    /* number of collected objects */
-    Py_ssize_t collected;
-    /* total number of uncollectable objects (put into gc.garbage) */
-    Py_ssize_t uncollectable;
-    // Total number of objects considered for collection and traversed:
-    Py_ssize_t candidates;
-    // Duration of the collection in seconds:
-    double duration;
-};
-
 /* Running stats per generation */
 struct gc_generation_stats {
+    PyTime_t ts_start;
+    PyTime_t ts_stop;
+
+    /* heap_size on the start of the collection */
+    Py_ssize_t heap_size;
+
+    /* work_to_do on the start of the collection */
+    Py_ssize_t work_to_do;
+
     /* total number of collections */
     Py_ssize_t collections;
+
+    /* total number of visited objects */
+    Py_ssize_t object_visits;
+
     /* total number of collected objects */
     Py_ssize_t collected;
     /* total number of uncollectable objects (put into gc.garbage) */
     Py_ssize_t uncollectable;
     // Total number of objects considered for collection and traversed:
     Py_ssize_t candidates;
-    // Duration of the collection in seconds:
+
+    Py_ssize_t objects_transitively_reachable;
+    Py_ssize_t objects_not_transitively_reachable;
+
+    // Total duration of the collection in seconds:
     double duration;
 };
 
+#ifdef Py_GIL_DISABLED
+#define GC_YOUNG_STATS_SIZE 1
+#define GC_OLD_STATS_SIZE 1
+#else
+#define GC_YOUNG_STATS_SIZE 11
+#define GC_OLD_STATS_SIZE 3
+#endif
+struct gc_young_stats_buffer {
+    struct gc_generation_stats items[GC_YOUNG_STATS_SIZE];
+    int8_t index;
+};
+
+struct gc_old_stats_buffer {
+    struct gc_generation_stats items[GC_OLD_STATS_SIZE];
+    int8_t index;
+};
+
 enum _GCPhase {
     GC_PHASE_MARK = 0,
     GC_PHASE_COLLECT = 1
@@ -211,6 +234,11 @@ enum _GCPhase {
    signature of gc.collect and change the size of PyStats.gc_stats */
 #define NUM_GENERATIONS 3
 
+struct gc_stats {
+    struct gc_young_stats_buffer young;
+    struct gc_old_stats_buffer old[2];
+};
+
 struct _gc_runtime_state {
     /* Is automatic collection enabled? */
     int enabled;
@@ -220,7 +248,7 @@ struct _gc_runtime_state {
     struct gc_generation old[2];
     /* a permanent generation which won't be collected */
     struct gc_generation permanent_generation;
-    struct gc_generation_stats generation_stats[NUM_GENERATIONS];
+    struct gc_stats generation_stats;
     /* true if we are currently running the collector */
     int collecting;
     // The frame that started the current collection. It might be NULL even when

Lib/_pyrepl/pager.py

@@ -138,7 +138,7 @@ def pipe_pager(text: str, cmd: str, title: str = '') -> None:
         '.'
         '?e (END):?pB %pB\\%..'
         ' (press h for help or q to quit)')
-    env['LESS'] = '-RmPm{0}$PM{0}$'.format(prompt_string)
+    env['LESS'] = '-RcmPm{0}$PM{0}$'.format(prompt_string)
     proc = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE,
                             errors='backslashreplace', env=env)
     assert proc.stdin is not None