I am using Flink 1.12.0, and I am reading https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/streaming/joins.html#processing-time-temporal-join. It looks that processing time temporal join is supported.
But, when I run the following application, it complains Processing-time temporal join is not supported yet. I am confused about whether it is code error or Flink really doesn't support Processing-time temporal join.
val env = StreamExecutionEnvironment.getExecutionEnvironment
env.setParallelism(1)
val tenv = StreamTableEnvironment.create(env)
val ddl1 =
"""
create table s1(
id STRING,
tradeDate TIMESTAMP,
price DOUBLE,
pt as PROCTIME()
) with (
'connector' = 'filesystem',
'path' = 'D:/T018_LookupJoin_stock.csv',
'format' = 'csv'
)
""".stripMargin(' ')
tenv.executeSql(ddl1)
val ddl2 =
"""
create table s2(
id STRING primary key not enforced,
name STRING,
tradeDate TIMESTAMP
) with (
'connector' = 'filesystem',
'path' = 'D:/T018_LookupJoin_stocktimechanging.csv',
'format' = 'csv'
)
""".stripMargin(' ')
tenv.executeSql(ddl2)
val sql =
"""
select s1.id ,s1.price, s1.tradeDate, u.name, u.tradeDate
from s1 join s2 for SYSTEM_TIME as of s1.pt as u
on s1.id = u.id
""".stripMargin(' ')
tenv.sqlQuery(sql).toAppendStream[Row].print()
env.execute()
}
Related
I have a sql defined this way
CREATE TABLE raw_table
(
headers VARCHAR,
id VARCHAR,
type VARCHAR,
contentJson VARCHAR
) WITH (
'connector' = 'kafka',
'topic-pattern' = 'role__.+?',
'properties.bootstrap.servers' = 'localhost:29092,localhost:39092',
'properties.group.id' = 'role_local_1',
'scan.startup.mode' = 'earliest-offset',
'format' = 'json',
'properties.allow.auto.create.topics' = 'true',
'json.timestamp-format.standard' = 'ISO-8601',
'sink.parallelism' = '3'
);
create view ROLES_NORMALIZED as
(
select
JSON_VALUE(contentJson, '$.id') as id,
rr.type as type
from raw_table rr
);
CREATE VIEW ROLES_UPSERTS_V1 AS
(
SELECT *
FROM ROLES_NORMALIZED
WHERE type in ('ROLE_CREATED', 'ROLE_UPDATED')
);
CREATE VIEW ROLES_DELETED_V1 AS
(
SELECT org,
pod,
tenantId,
id,
modified,
modified as deleted,
event_timestamp
FROM ROLES_NORMALIZED
WHERE type in ('ROLES_DELETED')
);
-------
CREATE TABLE final_topic
(
event_timestamp TIMESTAMP_LTZ,
id VARCHAR,
name VARCHAR,
deleted TIMESTAMP_LTZ,
PRIMARY KEY (pod, org, id) NOT ENFORCED
) WITH (
'connector' = 'upsert-kafka',
'topic' = 'final_topic',
'properties.bootstrap.servers' = 'localhost:29092,localhost:39092',
'properties.group.id' = 'some-group-id',
'value.format' = 'json',
'key.format' = 'json',
'properties.allow.auto.create.topics' = 'true',
'properties.replication.factor' = '3',
'value.json.timestamp-format.standard' = 'ISO-8601',
'sink.parallelism' = '3'
);
INSERT INTO final_topic
select
GREATEST(r.event_timestamp, d.event_timestamp) as event_timestamp,
r.id,
r.name,
d.deleted
from ROLES_UPSERTS_V1 r
LEFT JOIN ROLES_DELETED_V1 d
ON r.id = d.id;
The final_topic is produces the result i want to see, which is join of ROLES_UPSERTS_V1 and ROLES_DELETED_V1.
I tried this by publishing records to role__.+? topic.
What I am observing is that final-topic has null values as well. This is emitted when even the changelog kind happens to be -d -(DELETE). I understand the purpose as to why this exist. (here its saying the original message needs to be deleted and new one will follow). But I dont want such null values in my final-topic just the desired final state is this possible ?
Alternate that I am trying is to use Kafka connector. But the joins does not seems to work, as i get an error saying org.apache.flink.table.api.TableException: Table sink 'default_catalog.default_database.final_topic' doesn't support consuming update and delete changes which is produced by node Join(joinType=[LeftOuterJoin]. I get error when i use view for ROLES_UPSERTS_V1 and ROLES_DELETED_V1 . But if i have these as tables (with kafka connector) only inner join works ( left join does not work).
If you don't want null values, you can consider buffering records before you flush the results to the Upsert Kafka connector. See https://nightlies.apache.org/flink/flink-docs-stable/docs/connectors/table/upsert-kafka/#sink-buffer-flush-max-rows for more details
Regarding your join, as outlined in the docs For streaming queries, the grammar of regular joins is the most flexible and allow for any kind of updating (insert, update, delete) input table. Since you're running a streaming query, a future change could mean that the result of your join os an update or a delete. However, the sink that you're trying to emit to does not support this, hence the error.
I'm trying to implement a event-time temporal join but I don't see any data being emitted from the join. I don't see any runtime exceptions either.
Flink Version: 1.13
Kafka topics have only 1 partition for now
Here's how I set it up:
I have an "append-only" DataStream (left input/probe side) which looks like the following:
{
"eventType": String,
"eventTime": LocalDateTime,
"eventId": String
}
So, I convert this datastream to a table before joining them:
var eventTable = tableEnv.fromDataStream(eventStream, Schema.newBuilder()
.column("eventId", DataTypes.STRING())
.column("eventTime", DataTypes.TIMESTAMP(3))
.column("eventType", DataTypes.STRING())
.watermark("eventTime", $("eventTime"))
.build());
Then, I have the "versioned table" (right input/build side) backed by Kafka (Debezium CDC changelog) which looks like the following:
CREATE TABLE metadata (
id VARCHAR,
eventMetadata VARCHAR,
origin_ts TIMESTAMP(3) METADATA FROM 'value.source.timestamp' VIRTUAL,
PRIMARY KEY (id) NOT ENFORCED,
WATERMARK FOR origin_ts AS origin_ts
) WITH (
'connector' = 'kafka',
'properties.bootstrap.servers' = 'SERVER_ADDR',
'properties.group.id' = 'SOME_GROUP',
'topic' = 'SOME_TOPIC',
'scan.startup.mode' = 'latest-offset',
'value.format' = 'debezium-json'
)
The join query looks like this:
SELECT e.eventId, e.eventTime, e.eventType, m.eventMetadata
FROM events_view AS e
JOIN metadata_view FOR SYSTEM_TIME AS OF e.eventTime AS m
ON e.eventId = m.id
Following some other post on here, I've set the source idle-timeout:
table.exec.source.idle-timeout -> 5
And, I've also tried setting IdlenessTime on the watermarks to make sure source doesn't back emitting the watermarks. At this point I can see watermarks being generated, but I still don't get any results. Everything just ends up sitting on the Temporal Join table.
So, the problem here was the syntax of the processing time temporal join. Here's how to fix this:
// register the metadata table as a temporal table func by specifying its watermark and primary-key attributes
var metadataHistory = tableEnv.from("metadata")
.createTemporalTableFunction($("proc_time"), $("id"));
tableEnv.createTemporarySystemFunction("metadata_view", metadataHistory);
// sql processing time temporal join
var temporalJoinResult = tableEnv.sqlQuery("SELECT" +
" e.eventId, e.eventType, e.eventTime, m.eventMetadata" +
" FROM events_view AS e," +
" LATERAL TABLE (metadata_view(t.procTime)) AS m" +
" WHERE e.eventId = m.id");
Here, proc_time on metadata needs to be declared within the table DDL like this,
CREATE TABLE metadata (
id VARCHAR,
eventMetadata VARCHAR,
proc_time as PROCTIME(),
PRIMARY KEY (id) NOT ENFORCED
) WITH (
'connector' = 'kafka',
'properties.bootstrap.servers' = 'SERVER_ADDR',
'properties.group.id' = 'SOME_GROUP',
'topic' = 'SOME_TOPIC',
'scan.startup.mode' = 'latest-offset',
'value.format' = 'debezium-json'
)
and while converting the datastream to table, assign the procTime there for that table as well like this,
var eventTable = tableEnv.fromDataStream(eventStream, Schema.newBuilder()
.column("eventId", DataTypes.STRING())
.column("eventTime", DataTypes.TIMESTAMP(3))
.column("eventType", DataTypes.STRING())
.columnByExpression("procTime", "PROCTIME()")
.build());
I am needing help converting a PostgreSQL query to MSSQ.
Below is what i have done so far but i am issuing with the function and array areas which i do not think are allowed in MS SQL.
Is there something that that i need to do change the function and looks the WHERE statement has an array in it too.
I have added the select statement for the #temp table but when i create the #temp table i am getting errors saying incorrect syntax
CREATE FUNCTION pm_aggregate_report
(
_facility_ids uuid[]
, _risk_ids uuid[] DEFAULT NULL::uuid[]
, _assignee_ids uuid[] DEFAULT NULL::uuid[]
, _start_date date DEFAULT NULL::date
, _end_date date DEFAULT NULL::date
)
RETURNS TABLE
(
facility character varying
, pm_id uuid, grouped_pm boolean
, risk_id uuid
, risk character varying
, pm_status_id uuid
, user_id uuid
, assignee text
, completed_by uuid
, total_labor bigint
)
CREATE TABLE #tmp_pm_aggregate
(
facility_id VARCHAR(126),
pm_id VARCHAR(126),
grouped_pm VARCHAR(126),
risk_id VARCHAR(126),
pm_status_id VARCHAR(126),
user_id VARCHAR(126),
completed_by VARCHAR(126)
)
SELECT DISTINCT
COALESCE(gp.facility_id, a.facility_id) as facility_id,
COALESCE(p.grouped_pm_id, p.id) as pm_id,
CASE WHEN p.grouped_pm_id IS NULL THEN false ELSE true END as grouped_pm,
COALESCE(gp.risk_id, a.risk_id) as risk_id,
COALESCE(gp.pm_status_id, p.pm_status_id) as pm_status_id,
COALESCE(gass.user_id, sass.user_id) as user_id,
COALESCE(gp.completed_by, p.completed_by) as completed_by
FROM pms p
JOIN assets a
ON p.asset_id = a.id
LEFT JOIN grouped_pms gp
ON p.grouped_pm_id = gp.id
LEFT JOIN assignees sass
ON p.id = sass.record_id
AND sass.type = 'single_pm'
LEFT JOIN assignees gass
ON p.grouped_pm_id = gass.record_id
AND gass.type = 'grouped_pm'
LEFT JOIN users u
ON (sass.user_id = u.id OR gass.user_id = u.id)
WHERE a.facility_id = ANY(_facility_ids)
AND NOT a.is_component
AND COALESCE(gp.pm_status_id, p.pm_status_id) in ('f9bdfc17-3bb5-4ec0-8477-24ef05ea3b9b', '06fc910c-3d07-4284-8f6e-8fb3873f5333')
AND COALESCE(gp.completion_date, p.completion_date) BETWEEN COALESCE(_start_date, '1/1/2000') AND COALESCE(_end_date, '1/1/3000')
AND COALESCE(gp.show_date, p.show_date) <= CURRENT_TIMESTAMP
AND COALESCE(gass.user_id, sass.user_id) IS NOT NULL
AND u.user_type_id != 'ec823d98-7023-4908-8006-2e33ddf2c11b'
AND (_risk_ids IS NULL OR COALESCE(gp.risk_id, a.risk_id) = ANY(_risk_ids)
AND (_assignee_ids IS NULL OR COALESCE(gass.user_id, sass.user_id) = ANY(_assignee_ids);
SELECT
f.name as facility,
t.pm_id,
t.grouped_pm,
t.risk_id,
r.name as risk,
t.pm_status_id,
t.user_id,
u.name_last + ', ' + u.name_first as assignee,
t.completed_by,
ISNULL(gwl.total_labor, swl.total_labor) as total_labor
FROM #tmp_pm_aggregate t
JOIN facilities f
ON t.facility_id = f.id
JOIN risks r
ON t.risk_id = r.id
JOIN users u
ON t.user_id = u.id
LEFT JOIN (SELECT wl.record_id, wl.user_id, SUM(wl.labor_time) as total_labor
FROM work_logs wl
WHERE wl.type = 'single_pm'
GROUP BY wl.record_id, wl.user_id) as swl
ON t.pm_id = swl.record_id
AND t.user_id = swl.user_id
AND t.grouped_pm = false
LEFT JOIN (SELECT wl.record_id, wl.user_id, SUM(wl.labor_time) as total_labor
FROM work_logs wl
WHERE wl.type = 'grouped_pm'
GROUP BY wl.record_id, wl.user_id) as gwl
ON t.pm_id = gwl.record_id
AND t.user_id = gwl.user_id
AND t.grouped_pm = true
ORDER BY facility,
assignee,
risk;
DROP TABLE #tmp_pm_aggregate;
You can create an inline Table Valued Function, and simply return a resultset from it. You do not need (and cannot use) a temp table, you do not declare the returned "rowset" shape.
For the array parameters, you can use a Table Type:
CREATE TYPE dbo.GuidList (value uniqueidentifier NOT NULL PRIMARY KEY);
Because the table parameters are actual tables, you must query them like this (NOT EXISTS (SELECT 1 FROM #risk_ids) OR ISNULL(gp.risk_id, a.risk_id) IN (SELECT r.value FROM #risk_ids))
The parameters must start with #
There is no boolean type, you must use bit
Always use deterministic date formats for literals. yyyymmdd works for dates. Do you need to take into account hours and minutes, because you haven't?
ISNULL generally performs better than COALESCE in SQL Server, as the compiler understands it better
You may want to pass a separate parameter showing whether you passed in anything for the optional table parameters
I suggest you look carefully at the actual query: why does it need DISTINCT? It performs poorly, and is usually a code-smell indicating poorly thought-out joins. Perhaps you need to combine the two joins on assignees, or perhaps you should use a row-numbering strategy somewhere.
CREATE FUNCTION dbo.pm_aggregate_report
(
#facility_ids dbo.GuidList
, #risk_ids dbo.GuidList
, #assignee_ids dbo.GuidList
, #start_date date
, #end_date date
)
RETURNS TABLE AS RETURN
SELECT DISTINCT -- why DISTINCT, perhaps rethink your joins
ISNULL(gp.facility_id, a.facility_id) as facility_id,
ISNULL(p.grouped_pm_id, p.id) as pm_id,
CASE WHEN p.grouped_pm_id IS NULL THEN CAST(0 AS bit) ELSE CAST(1 AS bit) END as grouped_pm,
ISNULL(gp.risk_id, a.risk_id) as risk_id,
ISNULL(gp.pm_status_id, p.pm_status_id) as pm_status_id,
ISNULL(gass.user_id, sass.user_id) as user_id,
ISNULL(gp.completed_by, p.completed_by) as completed_by
FROM pms p
JOIN assets a
ON p.asset_id = a.id
LEFT JOIN grouped_pms gp
ON p.grouped_pm_id = gp.id
LEFT JOIN assignees sass
ON p.id = sass.record_id
AND sass.type = 'single_pm'
LEFT JOIN assignees gass
ON p.grouped_pm_id = gass.record_id
AND gass.type = 'grouped_pm'
LEFT JOIN users u
ON (sass.user_id = u.id OR gass.user_id = u.id) -- is this doubling up your rows?
WHERE a.facility_id IN (SELECT f.value FROM #facility_ids f)
AND a.is_component = 0
AND ISNULL(gp.pm_status_id, p.pm_status_id) in ('f9bdfc17-3bb5-4ec0-8477-24ef05ea3b9b', '06fc910c-3d07-4284-8f6e-8fb3873f5333')
AND ISNULL(gp.completion_date, p.completion_date) BETWEEN ISNULL(#start_date, '20000101') AND ISNULL(#end_date, '30000101') -- perhaps use >= AND <
AND ISNULL(gp.show_date, p.show_date) <= CURRENT_TIMESTAMP
AND ISNULL(gass.user_id, sass.user_id) IS NOT NULL
AND u.user_type_id != 'ec823d98-7023-4908-8006-2e33ddf2c11b'
AND (NOT EXISTS (SELECT 1 FROM #risk_ids) OR ISNULL(gp.risk_id, a.risk_id) IN (SELECT r.value FROM #risk_ids))
AND (NOT EXISTS (SELECT 1 FROM #assignee_ids) OR ISNULL(gass.user_id, sass.user_id) IN (SELECT aid.value FROM #assignee_ids aid));
I am working on converting a SQL Server query to Snowflake SQL. The code creates a dynamic start and end date, creates the temp table, then inserts the data from the query into the temp table.
Here is the query. When I run individual sections separately, I can create the variables and the temp table and the actual query to pull data also works.
However, I have not been able find the correct syntax for the INSERT statement to run correctly. Any help would be appreciated.
SET StartDate = (SELECT DATEADD(month, -24, dim.MONTH_BEGIN_DT)
FROM HI_DB.STG_EPICCLARITY_PHS.DATE_DIMENSION dim
WHERE CAST(GETDATE() AS date) = dim.CALENDAR_DT);
SET EndDate = (SELECT dim2.MONTH_BEGIN_DT
FROM HI_DB.STG_EPICCLARITY_PHS.DATE_DIMENSION dim2
WHERE CAST(GETDATE() AS date) = dim2.CALENDAR_DT);
CREATE TEMPORARY TABLE HI_DB.STG_EPICCLARITY_PHS.A1C_Min_Max
(
PAT_ID VARCHAR(255),
TEST_CNT NUMERIC(18,0),
ORD_ID_MIN NUMERIC(18,0),
ORD_ID_MAX NUMERIC(18,0)
);
INSERT INTO HI_DB.STG_EPICCLARITY_PHS.A1C_Min_Max (PAT_ID, TEST_CNT, ORD_ID_MIN, ORD_ID_MAX)
VALUES
SELECT
oprc.PAT_ID,
COUNT(*) as "TEST_CNT",
MIN(oprc.ORDER_PROC_ID) as "ORD_ID_MIN",
MAX(oprc.ORDER_PROC_ID) as "ORD_ID_MAX"
FROM
HI_DB.STG_EPICCLARITY_PHS.ORDER_PROC oprc
JOIN
HI_DB.STG_EPICCLARITY_PHS.PAT_ENC enc ON oprc.PAT_ENC_CSN_ID = enc.PAT_ENC_CSN_ID
INNER JOIN
HI_DB.STG_EPICCLARITY_PHS.ZC_DISP_ENC_TYPE typ ON enc.ENC_TYPE_C = typ.DISP_ENC_TYPE_C
INNER JOIN
HI_DB.STG_EPICCLARITY_PHS.CLARITY_EAP eap ON oprc.PROC_ID = eap.PROC_ID
INNER JOIN
HI_DB.STG_EPICCLARITY_PHS.ORDER_RESULTS ordres ON oprc.ORDER_PROC_ID = ordres.ORDER_PROC_ID
WHERE
oprc.ORDERING_DATE BETWEEN $StartDate AND $EndDate
AND enc.CONTACT_DATE BETWEEN $StartDate AND $EndDate
AND enc.SERV_AREA_ID = 12288
AND oprc.proc_id IN (12298843, 12299371, 122127749, 10050764, 12018926, 12037733)
AND ordres.COMPONENT_ID = 1202098 -- USE COMPONENT_ID = 1005276 to get ESTIMATED AVERAGE GLUCOSE VALUE
AND LEN(ordres.ORD_VALUE) > 1
GROUP BY
oprc.PAT_ID
SELECT *
FROM HI_DB.STG_EPICCLARITY_PHS.A1C_Min_Max
Try the query without the values keyword.
INSERT INTO A1C_Min_Max (PAT_ID,TEST_CNT,ORD_ID_MIN,ORD_ID_MAX)
SELECT
oprc.PAT_ID,
count(*) "TEST_CNT",
MIN(oprc.ORDER_PROC_ID) "ORD_ID_MIN",
MAX(oprc.ORDER_PROC_ID) "ORD_ID_MAX"
FROM ORDER_PROC oprc
JOIN PAT_ENC enc
ON oprc.PAT_ENC_CSN_ID = enc.PAT_ENC_CSN_ID
INNER JOIN ZC_DISP_ENC_TYPE typ
ON enc.ENC_TYPE_C = typ.DISP_ENC_TYPE_C
INNER JOIN CLARITY_EAP eap
ON oprc.PROC_ID = eap.PROC_ID
INNER JOIN ORDER_RESULTS ordres
ON oprc.ORDER_PROC_ID = ordres.ORDER_PROC_ID
WHERE oprc.ORDERING_DATE BETWEEN $StartDate AND $EndDate
AND enc.CONTACT_DATE BETWEEN $StartDate AND $EndDate
AND enc.SERV_AREA_ID = 12288
AND oprc.proc_id IN (12298843,12299371,122127749,10050764,12018926,12037733)
AND ordres.COMPONENT_ID = 1202098 -- USE COMPONENT_ID = 1005276 to get ESTIMATED AVERAGE GLUCOSE VALUE
AND LEN(ordres.ORD_VALUE) > 1
GROUP BY
oprc.PAT_ID;
Insert
INSERT [ OVERWRITE ] INTO target_table [ ( target_col_name [ , ... ] ) ]
{ { VALUES ( { value | DEFAULT | NULL } [ , ... ] ) [ , ( ... ) ] } | query }
In a VALUES clause, you can specify the following:
value: Inserts the explicitly-specified value.
DEFAULT: Inserts the default value for the corresponding column in the target table.
NULL: Inserts a NULL value.
https://docs.snowflake.com/en/sql-reference/sql/insert.html#required-parameters
I am trying to pull a query in MSSQL to pull the if they exist 9 digits zipcodes out of the cus_address table. I am already pulling 5 digit zipcodes out of a table i created myself called dbo.WEST_PALM_ZIPS. I need to pull all the zips in my WEST_PALM_ZIPS table as well as the CUS_ADDRESS.POSTAL_CODE (any help is appreciated)
SELECT DISTINCT [Member Id] = Cust.MASTER_CUSTOMER_ID,[FirstName] = Cust.First_Name
[Lastname] = Cust.Last_Name,
[Email] = Cust.PRIMARY_EMAIL_ADDRESS,
[Zip Code] = addr.POSTAL_CODE
FROM ORDER_DETAIL OD
INNER JOIN CUSTOMER Cust
ON Cust.MASTER_CUSTOMER_ID = OD.SHIP_MASTER_CUSTOMER_ID
and Cust.SUB_CUSTOMER_ID = OD.SHIP_SUB_CUSTOMER_ID
and od.subsystem = 'MBR'
INNER JOIN CUS_ADDRESS Addr
ON Addr.CUS_ADDRESS_ID = OD.SHIP_ADDRESS_ID
and Addr.POSTAL_CODE in ( select POSTAL_CODE from [dbo].[WEST_PALM_ZIPS])
and Addr.COUNTRY_descr = 'United States'
and If(Len([Postal_Code])>5,Left([Postal_Code],5) & "-" &
Mid([Postal_Code],6),[Postal_Code])][0-9]
INNER JOIN CUS_ADDRESS_DETAIL AddrD
ON AddrD.MASTER_CUSTOMER_ID = OD.SHIP_MASTER_CUSTOMER_ID
and AddrD.SUB_CUSTOMER_ID = OD.SHIP_SUB_CUSTOMER_ID
and AddrD.CUS_ADDRESS_ID = OD.SHIP_ADDRESS_ID
and AddrD.ADDRESS_TYPE_CODE = OD.SHIP_ADDRESS_TYPE_CODE
I think this is what you're after - though please shout if I've misunderstood anything / if it doesn't work as you'd expected.
SELECT DISTINCT [Member Id] = Cust.MASTER_CUSTOMER_ID
,[FirstName] = Cust.First_Name
,[Lastname] = Cust.Last_Name
,[Email] = Cust.PRIMARY_EMAIL_ADDRESS
,[Zip Code] = addr.POSTAL_CODE
FROM ORDER_DETAIL OD
INNER JOIN CUSTOMER Cust
ON Cust.MASTER_CUSTOMER_ID = OD.SHIP_MASTER_CUSTOMER_ID
and Cust.SUB_CUSTOMER_ID = OD.SHIP_SUB_CUSTOMER_ID
and od.subsystem = 'MBR'
INNER JOIN CUS_ADDRESS Addr
ON Addr.CUS_ADDRESS_ID = OD.SHIP_ADDRESS_ID
and Addr.COUNTRY_descr = 'United States'
--check for 5 digit or 9 digit zip codes
and
(
--check for 5 digit zip codes based on contents of table
(
LEN(Addr.POSTAL_CODE) = 5
and Addr.POSTAL_CODE in
(
select POSTAL_CODE
from [dbo].[WEST_PALM_ZIPS] wpz
--if you want to be sure you only get the 5 digit codes from this table (though from your description I believe that's all this table contains?)
--where LEN(POSTAL_CODE) = 5
--and Addr.POSTAL_CODE like '[0-9][0-9][0-9][0-9][0-9]'
)
)
or -- check that it's a 9 digit code
(
LEN(Addr.POSTAL_CODE) = 10
and SUBSTRING(Addr.POSTAL_CODE,6,1) = '-'
and Addr.POSTAL_CODE like '[0-9][0-9][0-9][0-9][0-9]-[0-9][0-9][0-9][0-9]'
)
)
INNER JOIN CUS_ADDRESS_DETAIL AddrD
ON AddrD.MASTER_CUSTOMER_ID = OD.SHIP_MASTER_CUSTOMER_ID
and AddrD.SUB_CUSTOMER_ID = OD.SHIP_SUB_CUSTOMER_ID
and AddrD.CUS_ADDRESS_ID = OD.SHIP_ADDRESS_ID
and AddrD.ADDRESS_TYPE_CODE = OD.SHIP_ADDRESS_TYPE_CODE
See http://msdn.microsoft.com/en-us/library/ms187489(SQL.90).aspx for an explanation of the like statement / pattern matching used in the above example.