Skip to content

Commit a6ced25

Browse files
DiegoDAFclaudej-bennet
authored
Add support for -f/--file option to execute SQL from files (#1543)
* Add support for -f/--file option to execute SQL from files This commit adds support for the -f/--file option to pgcli, similar to psql's behavior. Users can now execute SQL commands from files and exit immediately after execution. Features: - Single file execution: pgcli -f file.sql - Multiple files: pgcli -f file1.sql -f file2.sql - Long form: pgcli --file file.sql - Files are executed sequentially - Pager is automatically disabled in file mode - Proper error handling and exit codes Tests included for all scenarios. Made with ❤️ and 🤖 Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * Fix style checks and assert on the \dt output The style gate started failing with ruff 0.15.x: the TimeoutExpired handlers bound `as e` without using it (F841), and step_see_command_output decoded cmd_output into `output` but never used it. The command-output step now asserts on the \dt column headers, which are rendered whether or not any tables exist, so it verifies the special command actually produced its listing. * Run files one statement at a time so \watch stays scoped get_watch_command()'s regex captures all the text before a \watch, so feeding the whole file to handle_watch_command made \watch repeat every statement in it. Files now go through sqlparse.split() and run statement by statement, like psql: \watch repeats only its own statement, and a bare \watch picks up the previous one through query_history. A failed statement stops the rest of the file unless on_error is RESUME, matching what the single-block execution already did through pgexecute.run(). 7 tests; 5 fail without the fix. * Cut backslash commands at their newline, like psql sqlparse.split only cuts at semicolons, so a metacommand followed by SQL on the next line traveled as one chunk and the metacommand swallowed the SQL. Interactively this never happens because the buffer submits as soon as it starts with a backslash; files now follow psql's rule: a backslash command spans only its own line, and the rest of the chunk goes back through the splitter. 3 tests; all fail without the fix. --------- Co-authored-by: DiegoDAF <DiegoDAF@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Irina Truong <637013+j-bennet@users.noreply.github.com>
1 parent 9da9c8d commit a6ced25

5 files changed

Lines changed: 393 additions & 1 deletion

File tree

changelog.rst

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,17 @@
11
Upcoming
22
========
33

4+
Features:
5+
---------
6+
* Add support for executing SQL commands from file and exit.
7+
* Command line option `-f` or `--file`.
8+
* Multiple files can be specified.
9+
* Files run one statement at a time, like psql, so a ``\watch`` only
10+
repeats its own statement (and a bare ``\watch`` re-runs the statement
11+
before it), instead of re-running the whole file. A backslash command
12+
spans only its own line, also like psql, so a metacommand followed by
13+
SQL on the next line does not swallow the SQL.
14+
415
Bug fixes:
516
----------
617
* Fix special commands being broken while explain mode (F5) is on. Every input

pgcli/main.py

Lines changed: 78 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1045,6 +1045,33 @@ def _check_ongoing_transaction_and_allow_quitting(self):
10451045
def run_cli(self):
10461046
logger = self.logger
10471047

1048+
# Handle file mode (-f flag) - similar to psql behavior
1049+
# Multiple -f options are executed sequentially
1050+
if hasattr(self, 'input_files') and self.input_files:
1051+
try:
1052+
for input_file in self.input_files:
1053+
logger.debug("Reading commands from file: %s", input_file)
1054+
with open(input_file, 'r', encoding='utf-8') as f:
1055+
file_content = f.read()
1056+
1057+
if file_content.strip():
1058+
logger.debug("Executing commands from file: %s", input_file)
1059+
# Statement by statement, like psql -f: \watch only
1060+
# repeats its own statement, not the whole file.
1061+
if not self._execute_statements(file_content):
1062+
break
1063+
1064+
except PgCliQuitError:
1065+
# Normal exit from quit command
1066+
sys.exit(0)
1067+
except Exception as e:
1068+
logger.error("Error executing command: %s", e)
1069+
logger.error("traceback: %r", traceback.format_exc())
1070+
click.secho(str(e), err=True, fg="red")
1071+
sys.exit(1)
1072+
# Exit successfully after executing all commands
1073+
sys.exit(0)
1074+
10481075
history_file = self.config["main"]["history_file"]
10491076
if history_file == "default":
10501077
history_file = config_location() + "history"
@@ -1121,6 +1148,43 @@ def handle_watch_command(self, text):
11211148
query = self.execute_command(text)
11221149

11231150
self.query_history.append(query)
1151+
return query
1152+
1153+
def _execute_statements(self, text):
1154+
r"""Run a block of SQL the way psql -f does: one statement at a time.
1155+
1156+
get_watch_command()'s regex captures ALL the text before a \watch, so
1157+
feeding a whole file to handle_watch_command would make \watch repeat
1158+
every statement in it. Splitting first keeps \watch scoped to its own
1159+
statement, and a bare \watch picks up the previous statement through
1160+
query_history, exactly like psql.
1161+
1162+
A backslash command spans only its own line, like in psql, so a
1163+
metacommand followed by SQL on the next line does not swallow the
1164+
SQL (sqlparse only cuts at semicolons).
1165+
1166+
Honors on_error: with STOP, the first failed statement stops the run.
1167+
Returns True when every statement succeeded.
1168+
"""
1169+
ok = True
1170+
statements = sqlparse.split(text)
1171+
while statements:
1172+
statement = statements.pop(0)
1173+
stripped = statement.strip()
1174+
if not stripped:
1175+
continue
1176+
if stripped.startswith("\\") and "\n" in stripped:
1177+
# psql's rule: a backslash command ends at its newline. Put
1178+
# the rest back through the splitter.
1179+
first_line, rest = stripped.split("\n", 1)
1180+
statements = sqlparse.split(rest) + statements
1181+
statement = first_line
1182+
query = self.handle_watch_command(statement)
1183+
if query is not None and not query.successful:
1184+
ok = False
1185+
if self.on_error != "RESUME":
1186+
break
1187+
return ok
11241188

11251189
def _build_cli(self, history):
11261190
key_bindings = pgcli_bindings(self)
@@ -1426,7 +1490,8 @@ def is_too_tall(self, lines):
14261490
return len(lines) >= (self.prompt_app.output.get_size().rows - 4)
14271491

14281492
def echo_via_pager(self, text, color=None):
1429-
if self.pgspecial.pager_config == PAGER_OFF or self.watch_command:
1493+
# Disable pager for -f/--file mode and \watch command
1494+
if self.pgspecial.pager_config == PAGER_OFF or self.watch_command or (hasattr(self, 'input_files') and self.input_files):
14301495
click.echo(text, color=color)
14311496
elif self.pgspecial.pager_config == PAGER_LONG_OUTPUT and self.table_format != "csv":
14321497
lines = text.split("\n")
@@ -1581,6 +1646,14 @@ def echo_via_pager(self, text, color=None):
15811646
type=str,
15821647
help="SQL statement to execute after connecting.",
15831648
)
1649+
@click.option(
1650+
"-f",
1651+
"--file",
1652+
"input_files",
1653+
multiple=True,
1654+
type=click.Path(exists=True, readable=True, dir_okay=False),
1655+
help="execute commands from file, then exit. Multiple -f options are allowed.",
1656+
)
15841657
@click.argument("dbname", default=lambda: None, envvar="PGDATABASE", nargs=1)
15851658
@click.argument("username", default=lambda: None, envvar="PGUSER", nargs=1)
15861659
def cli(
@@ -1609,6 +1682,7 @@ def cli(
16091682
ssh_tunnel: str,
16101683
init_command: str,
16111684
log_file: str,
1685+
input_files: tuple,
16121686
connect_timeout: int | None,
16131687
):
16141688
if version:
@@ -1671,6 +1745,9 @@ def cli(
16711745
connect_timeout=connect_timeout,
16721746
)
16731747

1748+
# Store file paths for -f option (can be multiple)
1749+
pgcli.input_files = input_files if input_files else None
1750+
16741751
# Choose which ever one has a valid value.
16751752
if dbname_opt and dbname:
16761753
# work as psql: when database is given as option and argument use the argument as user

tests/features/file_option.feature

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
Feature: run the cli with -f/--file option,
2+
execute commands from file,
3+
and exit
4+
5+
Scenario: run pgcli with -f and a SQL query file
6+
When we create a file with "SELECT 1 as test_diego_column"
7+
and we run pgcli with -f and the file
8+
then we see the query result
9+
and pgcli exits successfully
10+
11+
Scenario: run pgcli with --file and a SQL query file
12+
When we create a file with "SELECT 'hello' as greeting"
13+
and we run pgcli with --file and the file
14+
then we see the query result
15+
and pgcli exits successfully
16+
17+
Scenario: run pgcli with -f and a file with special command
18+
When we create a file with "\dt"
19+
and we run pgcli with -f and the file
20+
then we see the command output
21+
and pgcli exits successfully
22+
23+
Scenario: run pgcli with -f and a file with multiple statements
24+
When we create a file with "SELECT 1; SELECT 2"
25+
and we run pgcli with -f and the file
26+
then we see both query results
27+
and pgcli exits successfully
28+
29+
Scenario: run pgcli with -f and a file with an invalid query
30+
When we create a file with "SELECT invalid_column FROM nonexistent_table"
31+
and we run pgcli with -f and the file
32+
then we see an error message
33+
and pgcli exits successfully
Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
"""
2+
Steps for testing -f/--file option behavioral tests.
3+
"""
4+
5+
import subprocess
6+
import tempfile
7+
import os
8+
from behave import when, then
9+
10+
11+
@when('we create a file with "{content}"')
12+
def step_create_file_with_content(context, content):
13+
"""Create a temporary file with the given content."""
14+
# Create a temporary file that will be cleaned up automatically
15+
temp_file = tempfile.NamedTemporaryFile(mode='w', delete=False, suffix='.sql')
16+
temp_file.write(content)
17+
temp_file.close()
18+
context.temp_file_path = temp_file.name
19+
20+
21+
@when('we run pgcli with -f and the file')
22+
def step_run_pgcli_with_f(context):
23+
"""Run pgcli with -f flag and the temporary file."""
24+
cmd = [
25+
"pgcli",
26+
"-h",
27+
context.conf["host"],
28+
"-p",
29+
str(context.conf["port"]),
30+
"-U",
31+
context.conf["user"],
32+
"-d",
33+
context.conf["dbname"],
34+
"-f",
35+
context.temp_file_path,
36+
]
37+
try:
38+
context.cmd_output = subprocess.check_output(cmd, cwd=context.package_root, stderr=subprocess.STDOUT, timeout=5)
39+
context.exit_code = 0
40+
except subprocess.CalledProcessError as e:
41+
context.cmd_output = e.output
42+
context.exit_code = e.returncode
43+
except subprocess.TimeoutExpired:
44+
context.cmd_output = b"Command timed out"
45+
context.exit_code = -1
46+
finally:
47+
# Clean up the temporary file
48+
if hasattr(context, 'temp_file_path') and os.path.exists(context.temp_file_path):
49+
os.unlink(context.temp_file_path)
50+
51+
52+
@when('we run pgcli with --file and the file')
53+
def step_run_pgcli_with_file(context):
54+
"""Run pgcli with --file flag and the temporary file."""
55+
cmd = [
56+
"pgcli",
57+
"-h",
58+
context.conf["host"],
59+
"-p",
60+
str(context.conf["port"]),
61+
"-U",
62+
context.conf["user"],
63+
"-d",
64+
context.conf["dbname"],
65+
"--file",
66+
context.temp_file_path,
67+
]
68+
try:
69+
context.cmd_output = subprocess.check_output(cmd, cwd=context.package_root, stderr=subprocess.STDOUT, timeout=5)
70+
context.exit_code = 0
71+
except subprocess.CalledProcessError as e:
72+
context.cmd_output = e.output
73+
context.exit_code = e.returncode
74+
except subprocess.TimeoutExpired:
75+
context.cmd_output = b"Command timed out"
76+
context.exit_code = -1
77+
finally:
78+
# Clean up the temporary file
79+
if hasattr(context, 'temp_file_path') and os.path.exists(context.temp_file_path):
80+
os.unlink(context.temp_file_path)
81+
82+
83+
@then("we see the query result")
84+
def step_see_query_result(context):
85+
"""Verify that the query result is in the output."""
86+
output = context.cmd_output.decode('utf-8')
87+
# Check for common query result indicators
88+
assert any([
89+
"SELECT" in output,
90+
"test_diego_column" in output,
91+
"greeting" in output,
92+
"hello" in output,
93+
"+-" in output, # table border
94+
"|" in output, # table column separator
95+
]), f"Expected query result in output, but got: {output}"
96+
97+
98+
@then("we see both query results")
99+
def step_see_both_query_results(context):
100+
"""Verify that both query results are in the output."""
101+
output = context.cmd_output.decode('utf-8')
102+
# Should contain output from both SELECT statements
103+
assert "SELECT" in output, f"Expected SELECT in output, but got: {output}"
104+
# The output should have multiple result sets
105+
assert output.count("SELECT") >= 2, f"Expected at least 2 SELECT results, but got: {output}"
106+
107+
108+
@then("we see the command output")
109+
def step_see_command_output(context):
110+
"""Verify that the special command output is present."""
111+
output = context.cmd_output.decode('utf-8')
112+
# `\dt` renders its column headers whether or not any tables exist, so the
113+
# headers are what tells us the special command actually ran and produced
114+
# its listing (rather than erroring out).
115+
for header in ("Schema", "Name", "Type", "Owner"):
116+
assert header in output, f"Expected {header!r} in \\dt output, but got: {output}"
117+
assert context.exit_code == 0, f"Expected exit code 0, but got: {context.exit_code}"
118+
119+
120+
@then("we see an error message")
121+
def step_see_error_message(context):
122+
"""Verify that an error message is in the output."""
123+
output = context.cmd_output.decode('utf-8')
124+
assert any([
125+
"does not exist" in output,
126+
"error" in output.lower(),
127+
"ERROR" in output,
128+
]), f"Expected error message in output, but got: {output}"
129+
130+
131+
@then("pgcli exits successfully")
132+
def step_pgcli_exits_successfully(context):
133+
"""Verify that pgcli exited with code 0."""
134+
assert context.exit_code == 0, f"Expected exit code 0, but got: {context.exit_code}"
135+
# Clean up
136+
context.cmd_output = None
137+
context.exit_code = None

0 commit comments

Comments
 (0)