postgres-xl.git
10 years agoMark a table in rowsecurity test case to be replicated
Pavan Deolasee [Mon, 22 Jun 2015 10:09:24 +0000 (15:39 +0530)]
Mark a table in rowsecurity test case to be replicated

The table is referenced by another table on a non-distribution key. So mark the
table as replicated

10 years agoAdjust out and read functions for TableSampleClause to honor portable out/read
Pavan Deolasee [Mon, 22 Jun 2015 10:07:36 +0000 (15:37 +0530)]
Adjust out and read functions for TableSampleClause to honor portable out/read

Instead of sending the OIDs, we need to send qualified names for various
objects in the structure

10 years agoFire remote DDLs correctly even if utility.c decides to route the statement via
Pavan Deolasee [Mon, 22 Jun 2015 05:06:27 +0000 (10:36 +0530)]
Fire remote DDLs correctly even if utility.c decides to route the statement via
ProcessUtilitySlow.

Even though bug existed only for GrantStmt, also refactor couple of the other
invocation points for code consistency

10 years agoDo not run queries starting from a remote coordinator inside a transaction
Pavan Deolasee [Fri, 19 Jun 2015 11:54:31 +0000 (17:24 +0530)]
Do not run queries starting from a remote coordinator inside a transaction
block

There are only a few occasions where a remote coordinator may need to connect
to datanodes, CREATE MATERIALIZED VIEW is one such example. We don't the remote
coordinator to run the SELECT query inside a transaction block. The DDL on the
remote coordinator is still done inside a transaction block and the SELECT uses
the global snapshot so the results will be consistent across all coordinators.
But using auto commit helps us running 2PC from the remote coordinator which
may confuse the datanodes

10 years agoDo not run REINDEX SCHEMA inside a transaction block
Pavan Deolasee [Fri, 19 Jun 2015 11:53:05 +0000 (17:23 +0530)]
Do not run REINDEX SCHEMA inside a transaction block

This matches with Postgres's newest behaviour

10 years agoHandle CREATE TABLE IF NOT EXISTS correctly
Pavan Deolasee [Thu, 18 Jun 2015 11:54:22 +0000 (17:24 +0530)]
Handle CREATE TABLE IF NOT EXISTS correctly

In XL we rewrite CTAS as CREATE TABLE followed by INSERT INTO. If the target
table already exists, we now issue a notice for the first step and skip the
second step completely. The ruleutils also now handles INE correctly for CTAS

10 years agoDo not propogate REINDEX command to coordinators for indexes on temporary
Pavan Deolasee [Thu, 18 Jun 2015 11:53:44 +0000 (17:23 +0530)]
Do not propogate REINDEX command to coordinators for indexes on temporary
tables

We don't create temp tables on the remote coordinators and hence we need not
(in fact, must not) send REINDEX command to those coordinators. Similarly, for
indexes on materialized views, REINDEX command should only be sent to the
coordinators. We now look at the underlying table properties and decide the
target node based on that. This is in sync with the CREATE TABLE/VIEW treatment
for these objects.

10 years agoAdd missing min/max(inet) aggregates.
Pavan Deolasee [Thu, 18 Jun 2015 09:08:04 +0000 (14:38 +0530)]
Add missing min/max(inet) aggregates.

They were most likely left out during the merge process. Also fix expected
output for the testcase 'inet' now that all issues are taken care of

10 years agoCreate installation directory before attempting to install GTM binaries
Pavan Deolasee [Thu, 18 Jun 2015 07:43:44 +0000 (13:13 +0530)]
Create installation directory before attempting to install GTM binaries

10 years agoDo not add pthread to LDFLAGS at the top level Makefile.
Pavan Deolasee [Wed, 17 Jun 2015 09:41:55 +0000 (15:11 +0530)]
Do not add pthread to LDFLAGS at the top level Makefile.

This was added in XC/XL, may be for building GTM, but this creates problems in
symbol resolution where the system library symbols are used over those
redefined in src/port. This patch fixes that issue

10 years agoDefine new routines for obtaining relation and tablespace path on the client
Pavan Deolasee [Mon, 15 Jun 2015 17:31:11 +0000 (23:01 +0530)]
Define new routines for obtaining relation and tablespace path on the client
side

10 years agoFix expected output for the testcase 'xc_having'
Pavan Deolasee [Wed, 10 Jun 2015 13:30:24 +0000 (19:00 +0530)]
Fix expected output for the testcase 'xc_having'

Most diffs were because of the plan showing the grouping key

10 years agoFix expected output for the testcase 'xc_groupby'
Pavan Deolasee [Wed, 10 Jun 2015 13:27:34 +0000 (18:57 +0530)]
Fix expected output for the testcase 'xc_groupby'

Most of the diffs were because of plan showing grouping key

10 years agoFix a few expected output diffs for the test case 'rowtypes'
Pavan Deolasee [Wed, 10 Jun 2015 13:04:10 +0000 (18:34 +0530)]
Fix a few expected output  diffs for the test case 'rowtypes'

10 years agoFix a few expected output diff for the testcase 'alter_table'
Pavan Deolasee [Wed, 10 Jun 2015 12:57:57 +0000 (18:27 +0530)]
Fix a few expected output diff for the testcase 'alter_table'

10 years agoFix alternate expected output for the testcase 'truncate'
Pavan Deolasee [Wed, 10 Jun 2015 12:54:25 +0000 (18:24 +0530)]
Fix alternate expected output for the testcase 'truncate'

We no longer emit NOTICE messages for implicit primary index creation

10 years agoFix expected output for the testcase 'domain'
Pavan Deolasee [Wed, 10 Jun 2015 12:53:33 +0000 (18:23 +0530)]
Fix expected output for the testcase 'domain'

10 years agoFix a few expected output diffs from the testcase 'equivclass'
Pavan Deolasee [Wed, 10 Jun 2015 11:10:37 +0000 (16:40 +0530)]
Fix a few expected output diffs from the testcase 'equivclass'

10 years agoFix expected output for the testcase 'indirect_toast'
Pavan Deolasee [Wed, 10 Jun 2015 11:07:47 +0000 (16:37 +0530)]
Fix expected output for the testcase 'indirect_toast'

10 years agoFix expected output file for the testcase 'window'
Pavan Deolasee [Wed, 10 Jun 2015 11:05:03 +0000 (16:35 +0530)]
Fix expected output file for the testcase 'window'

Apart from plan changes, we currently don't support ORDER BY in subqueries. So
handle those errors as expected messages

10 years agoFix alternate expected output file for the testcase 'foreign_data'
Pavan Deolasee [Wed, 10 Jun 2015 11:04:06 +0000 (16:34 +0530)]
Fix alternate expected output file for the testcase 'foreign_data'

10 years agoFix alternate expected output file for the testcase 'tsearch'
Pavan Deolasee [Wed, 10 Jun 2015 11:01:32 +0000 (16:31 +0530)]
Fix alternate expected output file for the testcase 'tsearch'

10 years agoFix alternate expected output file for the test case 'guc'
Pavan Deolasee [Wed, 10 Jun 2015 10:54:34 +0000 (16:24 +0530)]
Fix alternate expected output file for the test case 'guc'

10 years agoFix alternate expected output for the testcase 'cluster'
Pavan Deolasee [Wed, 10 Jun 2015 10:52:27 +0000 (16:22 +0530)]
Fix alternate expected output for the testcase 'cluster'

10 years agoFix alternate expected output file for testcase 'select_views'
Pavan Deolasee [Wed, 10 Jun 2015 10:48:59 +0000 (16:18 +0530)]
Fix alternate expected output file for testcase 'select_views'

10 years agoFix expected output for the testcase 'replica_identity'
Pavan Deolasee [Wed, 10 Jun 2015 10:28:51 +0000 (15:58 +0530)]
Fix expected output for the testcase 'replica_identity'

10 years agoFix a few expected output diffs in testcase 'matview'
Pavan Deolasee [Wed, 10 Jun 2015 10:27:45 +0000 (15:57 +0530)]
Fix a few expected output diffs in testcase 'matview'

There are more failures which need to be examined and fixed if needed

10 years agoFix expected output for the testcase 'collate'
Pavan Deolasee [Wed, 10 Jun 2015 10:25:42 +0000 (15:55 +0530)]
Fix expected output for the testcase 'collate'

Also merge changes from alternate expected output file with the original file
and remove the alternate expected output file

10 years agoFix expected output file for the testcase 'gist'
Pavan Deolasee [Wed, 10 Jun 2015 10:23:42 +0000 (15:53 +0530)]
Fix expected output file for the testcase 'gist'

10 years agoFix expected output file for testcase 'update'
Pavan Deolasee [Wed, 10 Jun 2015 10:21:35 +0000 (15:51 +0530)]
Fix expected output file for testcase 'update'

Continue to use alternate expected file and make changes to it

10 years agoFix expected output diffs from test case 'portals'
Pavan Deolasee [Wed, 10 Jun 2015 10:18:59 +0000 (15:48 +0530)]
Fix expected output diffs from test case 'portals'

The alternate expected output is changed, but we should consider merging those
changes to the original file in due course.

10 years agoFix expected output for test case 'transactions'
Pavan Deolasee [Wed, 10 Jun 2015 09:50:13 +0000 (15:20 +0530)]
Fix expected output for test case 'transactions'

10 years agoFix a few expected output diffs related to plan changes for test case
Pavan Deolasee [Wed, 10 Jun 2015 09:48:58 +0000 (15:18 +0530)]
Fix a few expected output diffs related to plan changes for test case
'aggregates'

10 years agoFix most expected output diffs in test case 'join'
Pavan Deolasee [Wed, 10 Jun 2015 09:40:06 +0000 (15:10 +0530)]
Fix most expected output diffs in test case 'join'

Most of these changes are related to plan changes. Also remove the alternate
expected output file and adjust the original file acoordingly

10 years agoFix alternate expected output file for test case 'case'
Pavan Deolasee [Wed, 10 Jun 2015 09:31:34 +0000 (15:01 +0530)]
Fix alternate expected output file for test case 'case'

10 years agoFix expected output diffs from test case union
Pavan Deolasee [Wed, 10 Jun 2015 08:52:22 +0000 (14:22 +0530)]
Fix expected output diffs from test case union

10 years agoFix a few diffs in expected output for test case subselect
Pavan Deolasee [Wed, 10 Jun 2015 08:50:24 +0000 (14:20 +0530)]
Fix a few diffs in expected output for test case subselect

These diffs are caused due to changes in the plan output

10 years agoFix expected output for test case sanity_check
Pavan Deolasee [Wed, 10 Jun 2015 08:46:18 +0000 (14:16 +0530)]
Fix expected output for test case sanity_check

This reflects new catalog tables created in XL. Also we fail to create indexes
on some tables (because of XL's limitations) and that is reflected in the
output

10 years agoFix expected output for test case vacuum by removing couple of additional
Pavan Deolasee [Wed, 10 Jun 2015 08:11:13 +0000 (13:41 +0530)]
Fix expected output for test case vacuum by removing couple of additional
contextual error messages

10 years agoFix a few expected output diffs from test case inherit
Pavan Deolasee [Wed, 10 Jun 2015 08:09:06 +0000 (13:39 +0530)]
Fix a few expected output diffs from test case inherit

Also removed alternate expected output file for the test case

10 years agoFix expected output diffs for test case triggers
Pavan Deolasee [Wed, 10 Jun 2015 07:44:30 +0000 (13:14 +0530)]
Fix expected output diffs for test case triggers

10 years agoFix a few expected output diffs in test case constraints
Pavan Deolasee [Wed, 10 Jun 2015 07:37:31 +0000 (13:07 +0530)]
Fix a few expected output diffs in test case constraints

10 years agoFix most of the expected output diffs for test case create_index
Pavan Deolasee [Wed, 10 Jun 2015 07:07:15 +0000 (12:37 +0530)]
Fix most of the expected output diffs for test case create_index

These diffs are either related to the plan changes because of RemoteQuery node
or because of the fact that PGXL does not yet support concurrent indexes. Other
failures need to be investigated

10 years agoFix a few expected outputs related to plan changes for test case
Pavan Deolasee [Wed, 10 Jun 2015 07:01:50 +0000 (12:31 +0530)]
Fix a few expected outputs related to plan changes for test case
insert_conflict.

There are a few more differences which need to be addressed

10 years agoFix expected output for test case macaddr
Pavan Deolasee [Wed, 10 Jun 2015 06:52:00 +0000 (12:22 +0530)]
Fix expected output for test case macaddr

10 years agoFix expected output for test case polygon
Pavan Deolasee [Wed, 10 Jun 2015 06:44:42 +0000 (12:14 +0530)]
Fix expected output for test case polygon

10 years agoFix expected output for rangetypes test case
Pavan Deolasee [Wed, 10 Jun 2015 06:21:47 +0000 (11:51 +0530)]
Fix expected output for rangetypes test case

10 years agoEncode list of rel OIDs in a portable format for datanodes to translate them to
Pavan Deolasee [Wed, 10 Jun 2015 06:17:02 +0000 (11:47 +0530)]
Encode list of rel OIDs in a portable format for datanodes to translate them to
local OIDs correctly

We now have a new routine to send down list of portable relation OIDs. Right
now only one member of ModifyTable node uses it, but we should see if this is
necessary at other places too.

10 years agoUse polymorphic inverse function for int8 aggregates.
Pavan Deolasee [Tue, 9 Jun 2015 06:16:55 +0000 (11:46 +0530)]
Use polymorphic inverse function for int8 aggregates.

We got it wrong during the merge process. Also enable window test case now
that the issue is resolved.

10 years agoEnable gist test case now that VACUUM hanging issue is resolved
Pavan Deolasee [Mon, 8 Jun 2015 11:43:41 +0000 (17:13 +0530)]
Enable gist test case now that VACUUM hanging issue is resolved

10 years agoRename sequences correctly on the GTM when schemas are renamed
Pavan Deolasee [Mon, 8 Jun 2015 11:41:15 +0000 (17:11 +0530)]
Rename sequences correctly on the GTM when schemas are renamed

We should do this for all non-temp sequences in the schema. The existing code
had a check other way round

10 years agoMerge remote-tracking branch 'remotes/PGSQL/master' into XL_NEW_MASTER
Pavan Deolasee [Fri, 5 Jun 2015 13:35:08 +0000 (19:05 +0530)]
Merge remote-tracking branch 'remotes/PGSQL/master' into XL_NEW_MASTER

Conflicts:
.gitignore
contrib/Makefile
src/backend/access/common/heaptuple.c
src/backend/access/transam/rmgr.c
src/backend/access/transam/xact.c
src/backend/catalog/Makefile
src/backend/catalog/catalog.c
src/backend/catalog/genbki.pl
src/backend/catalog/namespace.c
src/backend/commands/sequence.c
src/backend/executor/execMain.c
src/backend/executor/functions.c
src/backend/executor/nodeAgg.c
src/backend/executor/nodeModifyTable.c
src/backend/nodes/copyfuncs.c
src/backend/nodes/outfuncs.c
src/backend/nodes/readfuncs.c
src/backend/optimizer/plan/createplan.c
src/backend/optimizer/plan/planner.c
src/backend/optimizer/plan/setrefs.c
src/backend/optimizer/util/pathnode.c
src/backend/parser/gram.y
src/backend/parser/parse_agg.c
src/backend/parser/parse_utilcmd.c
src/backend/postmaster/postmaster.c
src/backend/replication/logical/decode.c
src/backend/storage/file/fd.c
src/backend/storage/ipc/procsignal.c
src/backend/tcop/utility.c
src/backend/utils/adt/lockfuncs.c
src/backend/utils/adt/ruleutils.c
src/backend/utils/sort/tuplesort.c
src/backend/utils/time/snapmgr.c
src/include/access/rmgrlist.h
src/include/catalog/pg_aggregate.h
src/include/catalog/pg_proc.h
src/include/nodes/execnodes.h
src/include/nodes/plannodes.h
src/include/nodes/primnodes.h
src/include/nodes/relation.h
src/include/storage/lwlock.h
src/include/storage/procsignal.h
src/include/utils/plancache.h
src/include/utils/snapshot.h
src/test/regress/expected/foreign_key.out
src/test/regress/expected/triggers.out
src/test/regress/expected/with.out
src/test/regress/input/constraints.source
src/test/regress/output/constraints.source
src/test/regress/pg_regress.c
src/test/regress/serial_schedule
src/test/regress/sql/returning.sql

10 years agoDo not call set_sid() for pooler processes.
Pavan Deolasee [Thu, 4 Jun 2015 10:22:39 +0000 (15:52 +0530)]
Do not call set_sid() for pooler processes.

A temp fix so that other tests can progress.

10 years agoFix some post-merge issues
Pavan Deolasee [Thu, 4 Jun 2015 10:22:13 +0000 (15:52 +0530)]
Fix some post-merge issues

10 years agoAdd missing columns for the newly added aggregate functions
Pavan Deolasee [Thu, 4 Jun 2015 08:40:06 +0000 (14:10 +0530)]
Add missing columns for the newly added aggregate functions

10 years agoMerge remote-tracking branch 'remotes/PGSQL/master' into XL_NEW_MASTER
Pavan Deolasee [Thu, 4 Jun 2015 08:02:02 +0000 (13:32 +0530)]
Merge remote-tracking branch 'remotes/PGSQL/master' into XL_NEW_MASTER

Conflicts:
COPYRIGHT
configure
configure.in
contrib/Makefile
doc/bug.template
src/backend/access/common/heaptuple.c
src/backend/access/common/printtup.c
src/backend/access/transam/Makefile
src/backend/access/transam/clog.c
src/backend/access/transam/twophase.c
src/backend/access/transam/varsup.c
src/backend/access/transam/xact.c
src/backend/access/transam/xlog.c
src/backend/bootstrap/bootstrap.c
src/backend/catalog/Makefile
src/backend/catalog/catalog.c
src/backend/catalog/dependency.c
src/backend/catalog/genbki.pl
src/backend/catalog/namespace.c
src/backend/catalog/pg_aggregate.c
src/backend/catalog/pg_proc.c
src/backend/catalog/storage.c
src/backend/commands/aggregatecmds.c
src/backend/commands/analyze.c
src/backend/commands/comment.c
src/backend/commands/copy.c
src/backend/commands/dbcommands.c
src/backend/commands/event_trigger.c
src/backend/commands/explain.c
src/backend/commands/indexcmds.c
src/backend/commands/portalcmds.c
src/backend/commands/schemacmds.c
src/backend/commands/sequence.c
src/backend/commands/tablecmds.c
src/backend/commands/trigger.c
src/backend/commands/vacuum.c
src/backend/commands/variable.c
src/backend/commands/view.c
src/backend/executor/execAmi.c
src/backend/executor/execCurrent.c
src/backend/executor/execMain.c
src/backend/executor/execProcnode.c
src/backend/executor/execTuples.c
src/backend/executor/execUtils.c
src/backend/executor/nodeAgg.c
src/backend/executor/nodeModifyTable.c
src/backend/executor/nodeSubplan.c
src/backend/executor/nodeWindowAgg.c
src/backend/libpq/hba.c
src/backend/nodes/copyfuncs.c
src/backend/nodes/equalfuncs.c
src/backend/nodes/outfuncs.c
src/backend/nodes/readfuncs.c
src/backend/optimizer/path/allpaths.c
src/backend/optimizer/path/costsize.c
src/backend/optimizer/plan/createplan.c
src/backend/optimizer/plan/planagg.c
src/backend/optimizer/plan/planner.c
src/backend/optimizer/plan/setrefs.c
src/backend/optimizer/plan/subselect.c
src/backend/optimizer/prep/preptlist.c
src/backend/optimizer/prep/prepunion.c
src/backend/optimizer/util/pathnode.c
src/backend/optimizer/util/plancat.c
src/backend/parser/analyze.c
src/backend/parser/gram.y
src/backend/parser/parse_agg.c
src/backend/parser/parse_relation.c
src/backend/parser/parse_utilcmd.c
src/backend/postmaster/autovacuum.c
src/backend/postmaster/pgstat.c
src/backend/postmaster/postmaster.c
src/backend/replication/logical/decode.c
src/backend/storage/buffer/bufmgr.c
src/backend/storage/ipc/ipci.c
src/backend/storage/ipc/procarray.c
src/backend/storage/ipc/procsignal.c
src/backend/storage/lmgr/lock.c
src/backend/storage/lmgr/lwlock.c
src/backend/storage/lmgr/proc.c
src/backend/tcop/dest.c
src/backend/tcop/postgres.c
src/backend/tcop/pquery.c
src/backend/tcop/utility.c
src/backend/utils/adt/arrayfuncs.c
src/backend/utils/adt/date.c
src/backend/utils/adt/dbsize.c
src/backend/utils/adt/pseudotypes.c
src/backend/utils/adt/ri_triggers.c
src/backend/utils/adt/ruleutils.c
src/backend/utils/adt/version.c
src/backend/utils/cache/inval.c
src/backend/utils/cache/lsyscache.c
src/backend/utils/cache/plancache.c
src/backend/utils/cache/relcache.c
src/backend/utils/init/globals.c
src/backend/utils/init/miscinit.c
src/backend/utils/init/postinit.c
src/backend/utils/misc/guc.c
src/backend/utils/mmgr/portalmem.c
src/backend/utils/sort/tuplesort.c
src/backend/utils/sort/tuplestore.c
src/backend/utils/time/combocid.c
src/backend/utils/time/snapmgr.c
src/bin/Makefile
src/bin/initdb/initdb.c
src/bin/pg_ctl/pg_ctl.c
src/bin/pg_dump/pg_dump.c
src/bin/pgbench/pgbench.c
src/bin/psql/tab-complete.c
src/include/access/htup.h
src/include/access/rmgrlist.h
src/include/access/transam.h
src/include/access/xact.h
src/include/catalog/catalog.h
src/include/catalog/namespace.h
src/include/catalog/pg_aggregate.h
src/include/catalog/pg_namespace.h
src/include/catalog/pg_proc.h
src/include/catalog/pg_type.h
src/include/commands/explain.h
src/include/commands/sequence.h
src/include/commands/vacuum.h
src/include/commands/variable.h
src/include/executor/execdesc.h
src/include/executor/executor.h
src/include/executor/tuptable.h
src/include/miscadmin.h
src/include/nodes/execnodes.h
src/include/nodes/nodes.h
src/include/nodes/params.h
src/include/nodes/parsenodes.h
src/include/nodes/plannodes.h
src/include/nodes/primnodes.h
src/include/nodes/relation.h
src/include/optimizer/cost.h
src/include/optimizer/pathnode.h
src/include/optimizer/planmain.h
src/include/parser/analyze.h
src/include/parser/parse_agg.h
src/include/parser/parse_utilcmd.h
src/include/pg_config.h.win32
src/include/pgstat.h
src/include/storage/backendid.h
src/include/storage/barrier.h
src/include/storage/lwlock.h
src/include/storage/proc.h
src/include/storage/procarray.h
src/include/storage/procsignal.h
src/include/storage/smgr.h
src/include/tcop/dest.h
src/include/tcop/pquery.h
src/include/utils/builtins.h
src/include/utils/guc.h
src/include/utils/lsyscache.h
src/include/utils/plancache.h
src/include/utils/portal.h
src/include/utils/rel.h
src/include/utils/tuplesort.h
src/include/utils/tuplestore.h
src/test/regress/expected/aggregates.out
src/test/regress/expected/create_index.out
src/test/regress/expected/foreign_data.out
src/test/regress/expected/join.out
src/test/regress/expected/macaddr.out
src/test/regress/expected/polygon.out
src/test/regress/expected/rangetypes.out
src/test/regress/expected/update.out
src/test/regress/input/constraints.source
src/test/regress/pg_regress.c
src/test/regress/serial_schedule
src/test/regress/sql/rangetypes.sql

10 years agoFix some issues in pg_class.relminmxid and pg_database.datminmxid documentation.
Fujii Masao [Thu, 4 Jun 2015 04:22:49 +0000 (13:22 +0900)]
Fix some issues in pg_class.relminmxid and pg_database.datminmxid documentation.

- Correct the name of directory which those catalog columns allow to be shrunk.
- Correct the name of symbol which is used as the value of pg_class.relminmxid
  when the relation is not a table.
- Fix "ID ID" typo.

Backpatch to 9.3 where those cataog columns were introduced.

10 years agodoc: Fix PDF build with FOP
Peter Eisentraut [Thu, 4 Jun 2015 00:19:47 +0000 (20:19 -0400)]
doc: Fix PDF build with FOP

Because of a bug in the DocBook XSL FO style sheet, an xref to a
varlistentry whose term includes an indexterm fails to build.  One such
instance was introduced in commit
5086dfceba79ecd5d1eb28b8f4ed5221838ff3a6.  Fix by adding the upstream
bug fix to our customization layer.

10 years agoFix some questionable edge-case behaviors in add_path() and friends.
Tom Lane [Wed, 3 Jun 2015 22:02:39 +0000 (18:02 -0400)]
Fix some questionable edge-case behaviors in add_path() and friends.

add_path_precheck was doing exact comparisons of path costs, but it really
needs to do them fuzzily to be sure it won't reject paths that could
survive add_path's comparisons.  (This can only matter if the initial cost
estimate is very close to the final one, but that turns out to often be
true.)

Also, it should ignore startup cost for this purpose if and only if
compare_path_costs_fuzzily would do so.  The previous coding always ignored
startup cost for parameterized paths, which is wrong as of commit
3f59be836c555fa6; it could result in improper early rejection of paths that
we care about for SEMI/ANTI joins.  It also always considered startup cost
for unparameterized paths, which is just as wrong though the only effect is
to waste planner cycles on paths that can't survive.  Instead, it should
consider startup cost only when directed to by the consider_startup/
consider_param_startup relation flags.

Likewise, compare_path_costs_fuzzily should have symmetrical behavior
for parameterized and unparameterized paths.  In this case, the best
answer seems to be that after establishing that total costs are fuzzily
equal, we should compare startup costs whether or not the consider_xxx
flags are on.  That is what it's always done for unparameterized paths,
so let's make the behavior for parameterized  paths match.

These issues were noted while developing the SEMI/ANTI join costing fix
of commit 3f59be836c555fa6, but we chose not to back-patch these fixes,
because they can cause changes in the planner's choices among
nearly-same-cost plans.  (There is in fact one minor change in plan choice
within the core regression tests.)  Destabilizing plan choices in back
branches without very clear improvements is frowned on, so we'll just fix
this in HEAD.

10 years agoFix planner's cost estimation for SEMI/ANTI joins with inner indexscans.
Tom Lane [Wed, 3 Jun 2015 15:58:47 +0000 (11:58 -0400)]
Fix planner's cost estimation for SEMI/ANTI joins with inner indexscans.

When the inner side of a nestloop SEMI or ANTI join is an indexscan that
uses all the join clauses as indexquals, it can be presumed that both
matched and unmatched outer rows will be processed very quickly: for
matched rows, we'll stop after fetching one row from the indexscan, while
for unmatched rows we'll have an indexscan that finds no matching index
entries, which should also be quick.  The planner already knew about this,
but it was nonetheless charging for at least one full run of the inner
indexscan, as a consequence of concerns about the behavior of materialized
inner scans --- but those concerns don't apply in the fast case.  If the
inner side has low cardinality (many matching rows) this could make an
indexscan plan look far more expensive than it actually is.  To fix,
rearrange the work in initial_cost_nestloop/final_cost_nestloop so that we
don't add the inner scan cost until we've inspected the indexquals, and
then we can add either the full-run cost or just the first tuple's cost as
appropriate.

Experimentation with this fix uncovered another problem: add_path and
friends were coded to disregard cheap startup cost when considering
parameterized paths.  That's usually okay (and desirable, because it thins
the path herd faster); but in this fast case for SEMI/ANTI joins, it could
result in throwing away the desired plain indexscan path in favor of a
bitmap scan path before we ever get to the join costing logic.  In the
many-matching-rows cases of interest here, a bitmap scan will do a lot more
work than required, so this is a problem.  To fix, add a per-relation flag
consider_param_startup that works like the existing consider_startup flag,
but applies to parameterized paths, and set it for relations that are the
inside of a SEMI or ANTI join.

To make this patch reasonably safe to back-patch, care has been taken to
avoid changing the planner's behavior except in the very narrow case of
SEMI/ANTI joins with inner indexscans.  There are places in
compare_path_costs_fuzzily and add_path_precheck that are not terribly
consistent with the new approach, but changing them will affect planner
decisions at the margins in other cases, so we'll leave that for a
HEAD-only fix.

Back-patch to 9.3; before that, the consider_startup flag didn't exist,
meaning that the second aspect of the patch would be too invasive.

Per a complaint from Peter Holzer and analysis by Tomas Vondra.

10 years agoMinor improvement to txid_current() documentation.
Fujii Masao [Wed, 3 Jun 2015 03:12:48 +0000 (12:12 +0900)]
Minor improvement to txid_current() documentation.

Michael Paquier, reviewed by Christoph Berg and Naoya Anzai

10 years agoInvalidate catalog snapshot upon starting a new global session
Pavan Deolasee [Tue, 2 Jun 2015 08:05:17 +0000 (13:35 +0530)]
Invalidate catalog snapshot upon starting a new global session

Also add a test case which demonstrates the problem of using wrong snapshot or
stale command id on datanode-to-datanode connection

10 years agoRelease notes for 9.4.3, 9.3.8, 9.2.12, 9.1.17, 9.0.21.
Tom Lane [Mon, 1 Jun 2015 17:27:43 +0000 (13:27 -0400)]
Release notes for 9.4.3, 9.3.8, 9.2.12, 9.1.17, 9.0.21.

Also sneak entries for commits 97ff2a564 et al into the sections for
the previous releases in the relevant branches.  Those fixes did go out
in the previous releases, but missed getting documented.

10 years agopgindent: add typedef blog URL
Bruce Momjian [Mon, 1 Jun 2015 15:27:30 +0000 (11:27 -0400)]
pgindent:  add typedef blog URL

10 years agoSupport snapshot requests without valid GXID
Pavan Deolasee [Mon, 1 Jun 2015 10:30:28 +0000 (16:00 +0530)]
Support snapshot requests without valid GXID

This allows callers to request snapshots without first obtaining a GXID from
the GTM. This should help read-only transactions.

In passing, also make some adjustments to the proxy code and add a check for
messages that are proxied between proxy and GTM. Otherwise a mistmatch between
message proxying and later response processing can cause hard-to-find bugs
(like one we srtuggled while creating this patch)

We still want some more enhancements so that a snapshot once obtained can be
recorded at the GTM so that subsequent requests can get the same snapshot, for
example for serializable transactions

10 years agoAdd missing support for handling extra configuration file for GTM proxies in
Pavan Deolasee [Mon, 1 Jun 2015 07:08:38 +0000 (12:38 +0530)]
Add missing support for handling extra configuration file for GTM proxies in
pgxc_ctl

10 years agoAvoid naming a variable "new", and remove bogus initializer.
Andrew Dunstan [Mon, 1 Jun 2015 02:56:53 +0000 (22:56 -0400)]
Avoid naming a variable "new", and remove bogus initializer.

Per gripe from Tom Lane.

10 years agoAdd a couple of missing JsonbValue type initialisers.
Andrew Dunstan [Mon, 1 Jun 2015 02:51:58 +0000 (22:51 -0400)]
Add a couple of missing JsonbValue type initialisers.

10 years agoRename jsonb_replace to jsonb_set and allow it to add new values
Andrew Dunstan [Mon, 1 Jun 2015 00:34:10 +0000 (20:34 -0400)]
Rename jsonb_replace to jsonb_set and allow it to add new values

The function is given a fourth parameter, which defaults to true. When
this parameter is true, if the last element of the path is missing
in the original json, jsonb_set creates it in the result and assigns it
the new value. If it is false then the function does nothing unless all
elements of the path are present, including the last.

Based on some original code from Dmitry Dolgov, heavily modified by me.

Catalog version bumped.

10 years agoMake Python tests more portable
Peter Eisentraut [Sun, 31 May 2015 11:10:45 +0000 (07:10 -0400)]
Make Python tests more portable

Newer Python versions randomize the hash seed for dictionaries,
resulting in a random output order, which messes up the regression test
diffs.

Instead, use Python assert to compare the dictionaries with their
expected value.

10 years agopg_upgrade: add missing period in C comment
Bruce Momjian [Fri, 29 May 2015 21:44:14 +0000 (17:44 -0400)]
pg_upgrade:  add missing period in C comment

10 years agoinitdb -S should now have an explicit check that $PGDATA is valid.
Tom Lane [Fri, 29 May 2015 21:02:58 +0000 (17:02 -0400)]
initdb -S should now have an explicit check that $PGDATA is valid.

The fsync code from the backend essentially assumes that somebody's already
validated PGDATA, at least to the extent of it being a readable directory.
That's safe enough for initdb's normal code path too, but "initdb -S"
doesn't have any other processing at all that touches the target directory.
To have reasonable error-case behavior, add a pg_check_dir call.
Per gripe from Peter E.

10 years agoRemove special cases for ETXTBSY from new fsync'ing logic.
Tom Lane [Fri, 29 May 2015 19:11:36 +0000 (15:11 -0400)]
Remove special cases for ETXTBSY from new fsync'ing logic.

The argument that this is a sufficiently-expected case to be silently
ignored seems pretty thin.  Andres had brought it up back when we were
still considering that most fsync failures should be hard errors, and it
probably would be legit not to fail hard for ETXTBSY --- but the same is
true for EROFS and other cases, which is why we gave up on hard failures.
ETXTBSY is surely not a normal case, so logging the failure seems fine
from here.

10 years agoCheck that all aliases of a built-in function have same leakproof property.
Tom Lane [Fri, 29 May 2015 17:26:21 +0000 (13:26 -0400)]
Check that all aliases of a built-in function have same leakproof property.

opr_sanity.sql has a test checking that relevant properties of built-in
functions match when the same C function is referenced by multiple pg_proc
entries.  The test neglected to check proleakproof, though, and when
I added that condition it exposed that xideqint4 hadn't been updated to
match xideq.  So fix that as well, and in consequence bump catversion.

This isn't very critical, so no need to worry about fixing back branches.

10 years agoAdjust initdb to also not consider fsync'ing failures fatal.
Tom Lane [Fri, 29 May 2015 17:05:16 +0000 (13:05 -0400)]
Adjust initdb to also not consider fsync'ing failures fatal.

Make initdb's version of this logic look as much like the backend's
as possible.  This is much less critical than in the backend since not
so many people use "initdb -S", but we want the same corner-case error
handling in both cases.

Back-patch to 9.3 where initdb -S option was introduced.  Before that,
initdb only had to deal with freshly-created data directories, wherein
no failures should be expected.

Abhijit Menon-Sen

10 years agoRevert exporting of internal GUC variable "data_directory".
Tom Lane [Fri, 29 May 2015 15:57:33 +0000 (11:57 -0400)]
Revert exporting of internal GUC variable "data_directory".

This undoes a poorly-thought-out choice in commit 970a18687f9b3058, namely
to export guc.c's internal variable data_directory.  The authoritative
variable so far as C code is concerned is DataDir; there is no reason for
anything except specific bits of GUC code to look at the GUC variable.

After yesterday's commits fixing the fsync-on-restart patch, the only
remaining misuse of data_directory was in AlterSystemSetConfigFile(),
which would be much better off just using a relative path anyhow: it's
less code and it doesn't break if the DBA moves the data directory of a
running system, which is a case we've taken some pains over in the past.

This is mostly cosmetic, so no need for a back-patch (and I'd be hesitant
to remove a global variable in stable branches anyway).

10 years agoRemove some other references of coord_name tracking in transaction info
Pavan Deolasee [Fri, 29 May 2015 11:09:09 +0000 (16:39 +0530)]
Remove some other references of coord_name tracking in transaction info
structure

10 years agoRemove coordinator name tracking from GTM transaction info structure
Pavan Deolasee [Fri, 29 May 2015 09:38:01 +0000 (15:08 +0530)]
Remove coordinator name tracking from GTM transaction info structure

This was not in use and hence removed

10 years agoSimplify GTM protocol a bit by only using GXID for transaction identification
Pavan Deolasee [Fri, 29 May 2015 07:59:01 +0000 (13:29 +0530)]
Simplify GTM protocol a bit by only using GXID for transaction identification

When the code was first written, it was assumed that client could directly use
index (handle) into an array to quickly identify a transaction. But that was
never used and was unnecessarily causing confusion. Remove that for simplification

10 years agoFix XL-only code in pgbench to reflect the changes in the upstream structure
Pavan Deolasee [Fri, 29 May 2015 07:55:21 +0000 (13:25 +0530)]
Fix XL-only code in pgbench to reflect the changes in the upstream structure

10 years agoFix fsync-at-startup code to not treat errors as fatal.
Tom Lane [Thu, 28 May 2015 21:33:03 +0000 (17:33 -0400)]
Fix fsync-at-startup code to not treat errors as fatal.

Commit 2ce439f3379aed857517c8ce207485655000fc8e introduced a rather serious
regression, namely that if its scan of the data directory came across any
un-fsync-able files, it would fail and thereby prevent database startup.
Worse yet, symlinks to such files also caused the problem, which meant that
crash restart was guaranteed to fail on certain common installations such
as older Debian.

After discussion, we agreed that (1) failure to start is worse than any
consequence of not fsync'ing is likely to be, therefore treat all errors
in this code as nonfatal; (2) we should not chase symlinks other than
those that are expected to exist, namely pg_xlog/ and tablespace links
under pg_tblspc/.  The latter restriction avoids possibly fsync'ing a
much larger part of the filesystem than intended, if the user has left
random symlinks hanging about in the data directory.

This commit takes care of that and also does some code beautification,
mainly moving the relevant code into fd.c, which seems a much better place
for it than xlog.c, and making sure that the conditional compilation for
the pre_sync_fname pass has something to do with whether pg_flush_data
works.

I also relocated the call site in xlog.c down a few lines; it seems a
bit silly to be doing this before ValidateXLOGDirectoryStructure().

The similar logic in initdb.c ought to be made to match this, but that
change is noncritical and will be dealt with separately.

Back-patch to all active branches, like the prior commit.

Abhijit Menon-Sen and Tom Lane

10 years agoRemove *pgaudit* references also.
Stephen Frost [Thu, 28 May 2015 17:02:09 +0000 (13:02 -0400)]
Remove *pgaudit* references also.

Fixes the docs build.

10 years agoFinish removing pg_audit
Stephen Frost [Thu, 28 May 2015 16:48:25 +0000 (12:48 -0400)]
Finish removing pg_audit

10 years agoFix pg_rewind's handling of top-level symlinks.
Tom Lane [Thu, 28 May 2015 16:44:31 +0000 (12:44 -0400)]
Fix pg_rewind's handling of top-level symlinks.

The previous coding suffered a null-pointer dereference if it found any
symlink at the top level of $PGDATA.  Fix that, and teach it to recurse
into a symlink for pg_xlog, but not anything else.

Per note from Abhijit Menon-Sen.

10 years agoRemove pg_audit
Stephen Frost [Thu, 28 May 2015 16:41:26 +0000 (12:41 -0400)]
Remove pg_audit

This removes pg_audit, per discussion:

20150528082038.GU26667@tamriel.snowman.net

10 years agoFix assorted inconsistencies in our calls of readlink().
Tom Lane [Thu, 28 May 2015 16:17:22 +0000 (12:17 -0400)]
Fix assorted inconsistencies in our calls of readlink().

Ensure that we null-terminate the result string (one place in pg_rewind).
Be paranoid about out-of-range results from readlink() (should not happen,
but there is no good reason for some call sites to be careful about it and
others not).  Consistently use the whole buffer, not sometimes one byte
less.  Ensure we emit an appropriate errcode() in all cases.  Spell the
error messages the same way.

The only serious bug here is the missing null-termination in pg_rewind,
which is new code, so no need for a back-patch.

Abhijit Menon-Sen and Tom Lane

10 years agoFix pg_get_functiondef() to print a function's LEAKPROOF property.
Tom Lane [Thu, 28 May 2015 15:24:37 +0000 (11:24 -0400)]
Fix pg_get_functiondef() to print a function's LEAKPROOF property.

Seems to have been an oversight in the original leakproofness patch.
Per report and patch from Jeevan Chalke.

In passing, prettify some awkward leakproof-related code in AlterFunction.

10 years agoAdd 'C' and 'R' to log_line_prefix.
Pavan Deolasee [Thu, 28 May 2015 10:48:05 +0000 (16:18 +0530)]
Add 'C' and 'R' to log_line_prefix.

This helps us print remote coordinator ID and PID of the remote coordinator
process. This is useful for debugging

10 years agoAvoid access to already-released lock in LockRefindAndRelease.
Robert Haas [Thu, 24 Jul 2014 12:19:19 +0000 (08:19 -0400)]
Avoid access to already-released lock in LockRefindAndRelease.

Spotted by Tom Lane.

10 years agoFix portability issue in isolationtester grammar.
Tom Lane [Wed, 27 May 2015 23:14:39 +0000 (19:14 -0400)]
Fix portability issue in isolationtester grammar.

specparse.y and specscanner.l used "string" as a token name.  Now, bison
likes to define each token name as a macro for the token code it assigns,
which means those names are basically off-limits for any other use within
the grammar file or included headers.  So names as generic as "string" are
dangerous.  This is what was causing the recent failures on protosciurus:
some versions of Solaris' sys/kstat.h use "string" as a field name.
With late-model bison we don't see this problem because the token macros
aren't defined till later (that is why castoroides didn't show the problem
even though it's on the same machine).  But protosciurus uses bison 1.875
which defines the token macros up front.

This land mine has been there from day one; we'd have found it sooner
except that protosciurus wasn't trying to run the isolation tests till
recently.

To fix, rename the token to "string_literal" which is hopefully less
likely to collide with names used by system headers.  Back-patch to
all branches containing the isolation tests.

10 years agoRevert "Add all structured objects passed to pushJsonbValue piecewise."
Andrew Dunstan [Wed, 27 May 2015 02:54:55 +0000 (22:54 -0400)]
Revert "Add all structured objects passed to pushJsonbValue piecewise."

This reverts commit 54547bd87f49326d67051254c363e6597d16ffda.

This appears to have been a thinko on my part. I will try to come up
wioth a better solution.

10 years agoRevert "Simplify addJsonbToParseState()"
Andrew Dunstan [Wed, 27 May 2015 02:54:11 +0000 (22:54 -0400)]
Revert "Simplify addJsonbToParseState()"

This reverts commit fba12c8c6c4159e1923958a4006b26f3cf873254.

This relied on a commit that is also being reverted.

10 years agoRemove configure check prohibiting threaded libpython on OpenBSD.
Tom Lane [Wed, 27 May 2015 02:14:59 +0000 (22:14 -0400)]
Remove configure check prohibiting threaded libpython on OpenBSD.

According to recent tests, this case now works fine, so there's no reason
to reject it anymore.  (Even if there are still some OpenBSD platforms
in the wild where it doesn't work, removing the check won't break any case
that worked before.)

We can actually remove the entire test that discovers whether libpython
is threaded, since without the OpenBSD case there's no need to know that
at all.

Per report from Davin Potts.  Back-patch to all active branches.

10 years agoSuppress occasional failures in brin regression test.
Tom Lane [Tue, 26 May 2015 18:10:46 +0000 (14:10 -0400)]
Suppress occasional failures in brin regression test.

brin.sql included a call of brin_summarize_new_values(), and expected
it to always report exactly 5 summarization events.  This failed sometimes
during parallel regression tests, as a consequence of the database-wide
VACUUM in gist.sql getting there first.  The most future-proof way
to avoid variation in the test results is to forget about using
brin_summarize_new_values() and just do a plain "VACUUM brintest",
which will exercise the same code anyway.

Having done that, there's no need for preventing autovacuum on brintest;
doing so just reduces the scope of test coverage, so let's not.

10 years agoSimplify addJsonbToParseState()
Andrew Dunstan [Tue, 26 May 2015 15:46:02 +0000 (11:46 -0400)]
Simplify addJsonbToParseState()

This function no longer needs to walk non-scalar structures passed to
it, following commit 54547bd87f49326d67051254c363e6597d16ffda.

10 years agoAdd all structured objects passed to pushJsonbValue piecewise.
Andrew Dunstan [Tue, 26 May 2015 15:16:52 +0000 (11:16 -0400)]
Add all structured objects passed to pushJsonbValue piecewise.

Commit 9b74f32cdbff8b9be47fc69164eae552050509ff did this for objects of
type jbvBinary, but in trying further to simplify some of the new jsonb
code I discovered that objects of type jbvObject or jbvArray passed as
WJB_ELEM or WJB_VALUE also caused problems. These too are now added
component by component.

Backpatch to 9.4.

10 years agoDo not run ALTER TYPE .. ADD in a transaction block on the datanode
Pavan Deolasee [Tue, 26 May 2015 08:59:24 +0000 (14:29 +0530)]
Do not run ALTER TYPE .. ADD in a transaction block on the datanode

10 years agoPass information about outer relids while creating subquery scan path.
Pavan Deolasee [Tue, 26 May 2015 06:28:27 +0000 (11:58 +0530)]
Pass information about outer relids while creating subquery scan path.

LATERAL queries were failing without this information correctly available

10 years agoFix valgrind's "unaddressable bytes" whining about BRIN code.
Tom Lane [Tue, 26 May 2015 01:56:19 +0000 (21:56 -0400)]
Fix valgrind's "unaddressable bytes" whining about BRIN code.

brin_form_tuple calculated an exact tuple size, then palloc'd and
filled just that much.  Later, brin_doinsert or brin_doupdate would
MAXALIGN the tuple size and tell PageAddItem that that was the size
of the tuple to insert.  If the original tuple size wasn't a multiple
of MAXALIGN, the net result would be that PageAddItem would memcpy
a few more bytes than the palloc request had been for.

AFAICS, this is totally harmless in the real world: the error is a
read overrun not a write overrun, and palloc would certainly have
rounded the request up to a MAXALIGN multiple internally, so there's
no chance of the memcpy fetching off the end of memory.  Valgrind,
however, is picky to the byte level not the MAXALIGN level.

Fix it by pushing the MAXALIGN step back to brin_form_tuple.  (The other
possible source of tuples in this code, brin_form_placeholder_tuple,
was already producing a MAXALIGN'd result.)

In passing, be a bit more paranoid about internal allocations in
brin_form_tuple.

10 years agopgindent: document location of "all" typedef lists
Bruce Momjian [Mon, 25 May 2015 20:53:48 +0000 (16:53 -0400)]
pgindent:  document location of "all" typedef lists