Skip to content

Commit 9770e32

Browse files
nessitaZackerySpytzerlend-aaslandgpshead
authored
gh-86533: Restore os.makedirs() ability to apply *mode* recursively (GH-150011)
bpo-42367: Restore os.makedirs() and pathlib.mkdir() ability to apply *mode* recursively via a new parent_mode= keyword argument. Co-authored-by: Zackery Spytz <zspytz@gmail.com> Co-authored-by: Erlend E. Aasland <erlend@python.org> Co-authored-by: Natalia <124304+nessita@users.noreply.github.com> Co-authored-by: Gregory P. Smith <greg@krypto.org>
1 parent 9eb50a4 commit 9770e32

8 files changed

Lines changed: 251 additions & 17 deletions

File tree

Doc/library/os.rst

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2549,7 +2549,8 @@ features:
25492549
Windows now handles a *mode* of ``0o700``.
25502550

25512551

2552-
.. function:: makedirs(name, mode=0o777, exist_ok=False)
2552+
.. function:: makedirs(name, mode=0o777, exist_ok=False, *, \
2553+
parent_mode=None)
25532554

25542555
.. index::
25552556
single: directory; creating
@@ -2567,6 +2568,12 @@ features:
25672568
If *exist_ok* is ``False`` (the default), a :exc:`FileExistsError` is
25682569
raised if the target directory already exists.
25692570

2571+
If *parent_mode* is not ``None``, it is used as the mode for any
2572+
newly-created, intermediate-level directories. Like *mode*, it is
2573+
combined with the process's umask value; see :ref:`the mkdir()
2574+
description <mkdir_modebits>`. Otherwise, intermediate directories are
2575+
created with the default mode, which is also subject to the umask.
2576+
25702577
.. note::
25712578

25722579
:func:`makedirs` will become confused if the path elements to create
@@ -2593,6 +2600,11 @@ features:
25932600
The *mode* argument no longer affects the file permission bits of
25942601
newly created intermediate-level directories.
25952602

2603+
.. versionadded:: 3.15
2604+
The *parent_mode* parameter. To match the behavior from Python 3.6 and
2605+
earlier (where *mode* was applied to all created directories), pass
2606+
``parent_mode=mode``.
2607+
25962608

25972609
.. function:: mkfifo(path, mode=0o666, *, dir_fd=None)
25982610

Doc/library/pathlib.rst

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1518,7 +1518,8 @@ Creating files and directories
15181518
:meth:`~Path.write_bytes` methods are often used to create files.
15191519

15201520

1521-
.. method:: Path.mkdir(mode=0o777, parents=False, exist_ok=False)
1521+
.. method:: Path.mkdir(mode=0o777, parents=False, exist_ok=False, *, \
1522+
parent_mode=None)
15221523

15231524
Create a new directory at this given path. If *mode* is given, it is
15241525
combined with the process's ``umask`` value to determine the file mode
@@ -1529,6 +1530,12 @@ Creating files and directories
15291530
as needed; they are created with the default permissions without taking
15301531
*mode* into account (mimicking the POSIX ``mkdir -p`` command).
15311532

1533+
If *parent_mode* is not ``None``, it is used as the mode for any
1534+
newly-created, intermediate-level directories when *parents* is true.
1535+
Like *mode*, it is combined with the process's ``umask`` value.
1536+
Otherwise, intermediate directories are created with the default
1537+
permissions (also subject to the umask).
1538+
15321539
If *parents* is false (the default), a missing parent raises
15331540
:exc:`FileNotFoundError`.
15341541

@@ -1542,6 +1549,9 @@ Creating files and directories
15421549
.. versionchanged:: 3.5
15431550
The *exist_ok* parameter was added.
15441551

1552+
.. versionadded:: 3.15
1553+
The *parent_mode* parameter.
1554+
15451555

15461556
.. method:: Path.symlink_to(target, target_is_directory=False)
15471557

Doc/whatsnew/3.15.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1285,6 +1285,10 @@ os
12851285
glibc versions 2.28 and later.
12861286
(Contributed by Jeffrey Bosboom and Victor Stinner in :gh:`83714`.)
12871287

1288+
* :func:`os.makedirs` function now has a *parent_mode* parameter that allows
1289+
specifying the mode for intermediate directories. This can be used to match
1290+
the behavior from Python 3.6 and earlier by passing ``parent_mode=mode``.
1291+
(Contributed by Zackery Spytz and Gregory P. Smith in :gh:`86533`.)
12881292

12891293
os.path
12901294
-------
@@ -2057,6 +2061,10 @@ importlib.resources
20572061
pathlib
20582062
-------
20592063

2064+
* :meth:`pathlib.Path.mkdir` now has a *parent_mode* parameter that allows
2065+
specifying the mode for intermediate directories when ``parents=True``.
2066+
(Contributed by Gregory P. Smith in :gh:`86533`.)
2067+
20602068
* Removed deprecated :meth:`!pathlib.PurePath.is_reserved`.
20612069
Use :func:`os.path.isreserved` to detect reserved paths on Windows.
20622070
(Contributed by Nikita Sobolev in :gh:`133875`.)

Lib/os.py

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -219,22 +219,29 @@ def _add(str, fn):
219219
# Super directory utilities.
220220
# (Inspired by Eric Raymond; the doc strings are mostly his)
221221

222-
def makedirs(name, mode=0o777, exist_ok=False):
223-
"""makedirs(name [, mode=0o777][, exist_ok=False])
222+
def makedirs(name, mode=0o777, exist_ok=False, *, parent_mode=None):
223+
"""makedirs(name [, mode=0o777][, exist_ok=False][, parent_mode=None])
224224
225225
Super-mkdir; create a leaf directory and all intermediate ones. Works like
226226
mkdir, except that any intermediate path segment (not just the rightmost)
227227
will be created if it does not exist. If the target directory already
228228
exists, raise an OSError if exist_ok is False. Otherwise no exception is
229-
raised. This is recursive.
229+
raised. If parent_mode is not None, it will be used as the mode for any
230+
newly-created, intermediate-level directories. Otherwise, intermediate
231+
directories are created with the default permissions (respecting umask).
232+
This is recursive.
230233
231234
"""
232235
head, tail = path.split(name)
233236
if not tail:
234237
head, tail = path.split(head)
235238
if head and tail and not path.exists(head):
236239
try:
237-
makedirs(head, exist_ok=exist_ok)
240+
if parent_mode is not None:
241+
makedirs(head, mode=parent_mode, exist_ok=exist_ok,
242+
parent_mode=parent_mode)
243+
else:
244+
makedirs(head, exist_ok=exist_ok)
238245
except FileExistsError:
239246
# Defeats race condition when another thread created the path
240247
pass

Lib/pathlib/__init__.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1204,7 +1204,7 @@ def touch(self, mode=0o666, exist_ok=True):
12041204
fd = os.open(self, flags, mode)
12051205
os.close(fd)
12061206

1207-
def mkdir(self, mode=0o777, parents=False, exist_ok=False):
1207+
def mkdir(self, mode=0o777, parents=False, exist_ok=False, *, parent_mode=None):
12081208
"""
12091209
Create a new directory at this given path.
12101210
"""
@@ -1213,7 +1213,11 @@ def mkdir(self, mode=0o777, parents=False, exist_ok=False):
12131213
except FileNotFoundError:
12141214
if not parents or self.parent == self:
12151215
raise
1216-
self.parent.mkdir(parents=True, exist_ok=True)
1216+
if parent_mode is not None:
1217+
self.parent.mkdir(mode=parent_mode, parents=True, exist_ok=True,
1218+
parent_mode=parent_mode)
1219+
else:
1220+
self.parent.mkdir(parents=True, exist_ok=True)
12171221
self.mkdir(mode, parents=False, exist_ok=exist_ok)
12181222
except OSError:
12191223
# Cannot rely on checking for EEXIST, since the operating system

Lib/test/test_os/test_os.py

Lines changed: 91 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -2139,6 +2139,94 @@ def test_mode(self):
21392139
self.assertEqual(os.stat(path).st_mode & 0o777, 0o555)
21402140
self.assertEqual(os.stat(parent).st_mode & 0o777, 0o775)
21412141

2142+
@unittest.skipIf(
2143+
support.is_emscripten or support.is_wasi,
2144+
"umask is not implemented on Emscripten/WASI."
2145+
)
2146+
@unittest.skipIf(
2147+
sys.platform == "android",
2148+
"Android filesystem may not honor requested permissions."
2149+
)
2150+
def test_mode_with_parent_mode(self):
2151+
# Test the parent_mode parameter
2152+
parent = os.path.join(os_helper.TESTFN, 'dir1')
2153+
path = os.path.join(parent, 'dir2')
2154+
with os_helper.temp_umask(0o002):
2155+
# Specify mode for both leaf and parent directories
2156+
os.makedirs(path, 0o770, parent_mode=0o750)
2157+
self.assertTrue(os.path.exists(path))
2158+
self.assertTrue(os.path.isdir(path))
2159+
if os.name != 'nt':
2160+
# Leaf directory gets the mode parameter
2161+
self.assertEqual(os.stat(path).st_mode & 0o777, 0o770)
2162+
# Parent directory gets the parent_mode parameter
2163+
self.assertEqual(os.stat(parent).st_mode & 0o777, 0o750)
2164+
2165+
@unittest.skipIf(
2166+
support.is_emscripten or support.is_wasi,
2167+
"umask is not implemented on Emscripten/WASI."
2168+
)
2169+
@unittest.skipIf(
2170+
sys.platform == "android",
2171+
"Android filesystem may not honor requested permissions."
2172+
)
2173+
def test_parent_mode_deep_hierarchy(self):
2174+
# Test parent_mode with deep directory hierarchy
2175+
base = os.path.join(os_helper.TESTFN, 'dir1', 'dir2', 'dir3')
2176+
with os_helper.temp_umask(0o002):
2177+
os.makedirs(base, 0o755, parent_mode=0o700)
2178+
self.assertTrue(os.path.exists(base))
2179+
if os.name != 'nt':
2180+
# Check that all parent directories have parent_mode
2181+
level1 = os.path.join(os_helper.TESTFN, 'dir1')
2182+
level2 = os.path.join(level1, 'dir2')
2183+
self.assertEqual(os.stat(level1).st_mode & 0o777, 0o700)
2184+
self.assertEqual(os.stat(level2).st_mode & 0o777, 0o700)
2185+
# Leaf directory has the regular mode
2186+
self.assertEqual(os.stat(base).st_mode & 0o777, 0o755)
2187+
2188+
@unittest.skipIf(
2189+
support.is_emscripten or support.is_wasi,
2190+
"umask is not implemented on Emscripten/WASI."
2191+
)
2192+
@unittest.skipIf(
2193+
sys.platform == "android",
2194+
"Android filesystem may not honor requested permissions."
2195+
)
2196+
def test_parent_mode_same_as_mode(self):
2197+
# Test emulating Python 3.6 behavior by setting parent_mode=mode
2198+
parent = os.path.join(os_helper.TESTFN, 'dir1')
2199+
path = os.path.join(parent, 'dir2')
2200+
with os_helper.temp_umask(0o002):
2201+
os.makedirs(path, 0o705, parent_mode=0o705)
2202+
self.assertTrue(os.path.exists(path))
2203+
if os.name != 'nt':
2204+
# Both directories should have the same mode
2205+
self.assertEqual(os.stat(path).st_mode & 0o777, 0o705)
2206+
self.assertEqual(os.stat(parent).st_mode & 0o777, 0o705)
2207+
2208+
@unittest.skipIf(
2209+
support.is_emscripten or support.is_wasi,
2210+
"umask is not implemented on Emscripten/WASI."
2211+
)
2212+
@unittest.skipIf(
2213+
sys.platform == "android",
2214+
"Android filesystem may not honor requested permissions."
2215+
)
2216+
def test_parent_mode_combined_with_umask(self):
2217+
# parent_mode, like mode, is combined with the process umask; it does
2218+
# not bypass it.
2219+
parent = os.path.join(os_helper.TESTFN, 'dir1')
2220+
path = os.path.join(parent, 'dir2')
2221+
with os_helper.temp_umask(0o022):
2222+
os.makedirs(path, 0o777, parent_mode=0o777)
2223+
self.assertTrue(os.path.isdir(path))
2224+
if os.name != 'nt':
2225+
# 0o777 is masked down to 0o755 by the 0o022 umask, for both
2226+
# the leaf (mode) and the parent (parent_mode).
2227+
self.assertEqual(os.stat(path).st_mode & 0o777, 0o755)
2228+
self.assertEqual(os.stat(parent).st_mode & 0o777, 0o755)
2229+
21422230
@unittest.skipIf(
21432231
support.is_wasi,
21442232
"WASI's umask is a stub."
@@ -2212,15 +2300,9 @@ def test_win32_mkdir_700(self):
22122300
)
22132301

22142302
def tearDown(self):
2215-
path = os.path.join(os_helper.TESTFN, 'dir1', 'dir2', 'dir3',
2216-
'dir4', 'dir5', 'dir6')
2217-
# If the tests failed, the bottom-most directory ('../dir6')
2218-
# may not have been created, so we look for the outermost directory
2219-
# that exists.
2220-
while not os.path.exists(path) and path != os_helper.TESTFN:
2221-
path = os.path.dirname(path)
2222-
2223-
os.removedirs(path)
2303+
# Remove the whole tree regardless of which sub-directories a test
2304+
# created and regardless of their permission bits.
2305+
os_helper.rmtree(os_helper.TESTFN)
22242306

22252307

22262308
@unittest.skipUnless(hasattr(os, "chown"), "requires os.chown()")

Lib/test/test_pathlib/test_pathlib.py

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2492,6 +2492,113 @@ def my_mkdir(path, mode=0o777):
24922492
self.assertNotIn(str(p12), concurrently_created)
24932493
self.assertTrue(p.exists())
24942494

2495+
@unittest.skipIf(
2496+
is_emscripten or is_wasi,
2497+
"umask is not implemented on Emscripten/WASI."
2498+
)
2499+
@unittest.skipIf(
2500+
sys.platform == "android",
2501+
"Android filesystem may not honor requested permissions."
2502+
)
2503+
def test_mkdir_parents_umask(self):
2504+
# Test that parent directories respect umask when parent_mode is not set
2505+
p = self.cls(self.base, 'umasktest', 'child')
2506+
self.assertFalse(p.exists())
2507+
if os.name != 'nt':
2508+
with os_helper.temp_umask(0o002):
2509+
p.mkdir(0o755, parents=True)
2510+
self.assertTrue(p.exists())
2511+
# Leaf directory gets the specified mode
2512+
self.assertEqual(p.stat().st_mode & 0o777, 0o755)
2513+
# Parent directory respects umask (0o777 & ~0o002 = 0o775)
2514+
self.assertEqual(p.parent.stat().st_mode & 0o777, 0o775)
2515+
2516+
@unittest.skipIf(
2517+
is_emscripten or is_wasi,
2518+
"umask is not implemented on Emscripten/WASI."
2519+
)
2520+
@unittest.skipIf(
2521+
sys.platform == "android",
2522+
"Android filesystem may not honor requested permissions."
2523+
)
2524+
def test_mkdir_with_parent_mode(self):
2525+
# Test the parent_mode parameter
2526+
p = self.cls(self.base, 'newdirPM', 'subdirPM')
2527+
self.assertFalse(p.exists())
2528+
if os.name != 'nt':
2529+
# Specify different modes for parent and leaf directories
2530+
p.mkdir(0o755, parents=True, parent_mode=0o750)
2531+
self.assertTrue(p.exists())
2532+
self.assertTrue(p.is_dir())
2533+
# Leaf directory gets the mode parameter
2534+
self.assertEqual(p.stat().st_mode & 0o777, 0o755)
2535+
# Parent directory gets the parent_mode parameter
2536+
self.assertEqual(p.parent.stat().st_mode & 0o777, 0o750)
2537+
2538+
@unittest.skipIf(
2539+
is_emscripten or is_wasi,
2540+
"umask is not implemented on Emscripten/WASI."
2541+
)
2542+
@unittest.skipIf(
2543+
sys.platform == "android",
2544+
"Android filesystem may not honor requested permissions."
2545+
)
2546+
def test_mkdir_parent_mode_deep_hierarchy(self):
2547+
# Test parent_mode with deep directory hierarchy
2548+
p = self.cls(self.base, 'level1PM', 'level2PM', 'level3PM')
2549+
self.assertFalse(p.exists())
2550+
if os.name != 'nt':
2551+
p.mkdir(0o755, parents=True, parent_mode=0o700)
2552+
self.assertTrue(p.exists())
2553+
# Check that all parent directories have parent_mode
2554+
level1 = self.cls(self.base, 'level1PM')
2555+
level2 = level1 / 'level2PM'
2556+
self.assertEqual(level1.stat().st_mode & 0o777, 0o700)
2557+
self.assertEqual(level2.stat().st_mode & 0o777, 0o700)
2558+
# Leaf directory has the regular mode
2559+
self.assertEqual(p.stat().st_mode & 0o777, 0o755)
2560+
2561+
@unittest.skipIf(
2562+
is_emscripten or is_wasi,
2563+
"umask is not implemented on Emscripten/WASI."
2564+
)
2565+
@unittest.skipIf(
2566+
sys.platform == "android",
2567+
"Android filesystem may not honor requested permissions."
2568+
)
2569+
def test_mkdir_parent_mode_combined_with_umask(self):
2570+
# parent_mode, like mode, is combined with the process umask; it does
2571+
# not bypass it.
2572+
p = self.cls(self.base, 'umaskPM', 'child')
2573+
self.assertFalse(p.exists())
2574+
if os.name != 'nt':
2575+
with os_helper.temp_umask(0o022):
2576+
p.mkdir(0o777, parents=True, parent_mode=0o777)
2577+
self.assertTrue(p.exists())
2578+
# 0o777 is masked down to 0o755 by the 0o022 umask, for both
2579+
# the leaf (mode) and the parent (parent_mode).
2580+
self.assertEqual(p.stat().st_mode & 0o777, 0o755)
2581+
self.assertEqual(p.parent.stat().st_mode & 0o777, 0o755)
2582+
2583+
@unittest.skipIf(
2584+
is_emscripten or is_wasi,
2585+
"umask is not implemented on Emscripten/WASI."
2586+
)
2587+
@unittest.skipIf(
2588+
sys.platform == "android",
2589+
"Android filesystem may not honor requested permissions."
2590+
)
2591+
def test_mkdir_parent_mode_same_as_mode(self):
2592+
# Test setting parent_mode same as mode
2593+
p = self.cls(self.base, 'samedirPM', 'subdirPM')
2594+
self.assertFalse(p.exists())
2595+
if os.name != 'nt':
2596+
p.mkdir(0o705, parents=True, parent_mode=0o705)
2597+
self.assertTrue(p.exists())
2598+
# Both directories should have the same mode
2599+
self.assertEqual(p.stat().st_mode & 0o777, 0o705)
2600+
self.assertEqual(p.parent.stat().st_mode & 0o777, 0o705)
2601+
24952602
@needs_symlinks
24962603
def test_symlink_to(self):
24972604
P = self.cls(self.base)
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
The :func:`os.makedirs` function and :meth:`pathlib.Path.mkdir` method now have
2+
a *parent_mode* parameter to specify the mode for intermediate directories when
3+
creating parent directories. This allows one to match the behavior from Python
4+
3.6 and earlier for :func:`os.makedirs`.

0 commit comments

Comments
 (0)