作者:Wx丶华少 | 来源:互联网 | 2023-09-16 09:19
TL:DR;Isitpossibletocreateacustomtriggerthatonlyfiresifsomeflagisset?Isitpossible
TL:DR; Is it possible to create a custom trigger that only fires if some flag is set? Is it possible to deploy the job with a trigger with a huge delay while we know a large data event is happening, and then deploy an update to the job with the trigger having a normal or no delay once that event is finished?
TL:DR;是否可以创建自定义触发器,仅在设置了某个标志时触发?当我们知道正在发生大数据事件时,是否可以使用具有巨大延迟的触发器部署作业,然后在事件结束后将触发器部署到具有正常或无延迟的触发器的更新?
Following on from: Remove duplicates across window triggers/firings
继续之后:删除窗口触发器/触发之间的重复项
The situation where this happens the most problematically (millions of duplicate firings) is when we're doing a backfill of old data. Given we know when this was happening I was wondering if we could implement a custom trigger that doesn't fire while a flag is set. Is that something that would be possible? Alternatively, could we deploy the job with a trigger that includes a huge delay while backfill is going on, and then issue an update with the normal trigger when it's finished?
这种情况发生最有问题(数百万次重复点火)的情况是我们正在对旧数据进行回填。鉴于我们知道何时发生这种情况,我想知道我们是否可以实现一个自定义触发器,该触发器在设置标志时不会触发。那是可能的吗?或者,我们是否可以使用触发器部署作业,该触发器在回填过程中包含大量延迟,然后在完成时使用正常触发器发出更新?
1 个解决方案